According to The Times, Gary was charged after his injured wife was reportedly discovered by a taxi driver on the street outside their mansion in Marylebone, London.
It's claimed the incident occurred around 1:30am on Friday morning as the couple returned from a charity auction event at Home House in London.
Taxi driver Daniel Shepherd told The Sun Gary struck his wife in the eye after she accused him of doing cocaine.
“He was screaming and swearing at her and she slapped him round the face and his glasses fell off," he claimed.
“He then turned and hit her in the face with a left hook. She fell and cracked her head on the pavement.”
"Off to @HomeHouseLondon for the annual @lastnightadj charity gig with @cheechkeela @RussClem65 et al. Always a great night- dang this diet!" Gary, who is Carole Middleton's younger brother, tweeted before the event.
Gary was released on bail and will have to appear at Westminster magistrates’ court on October 31.
The Daily Mail reports that Gary, who has been married three times before tying the knot to fourth wife Julie-Ann, has been banned from speaking to his partner while the matter is under investigating.
It was only in Mary that the Middleton's had extended an olive branch and welcomed him back into the family fold with an invitation to Pippa's wedding ceremony to James Matthews.
This isn't the first time the "black sheep" of Kate's family has made headlines.
After being exposed as a playboy involved in hard drugs in 2009 who proudly bragged about his royal connections, Prince William cut ties with Mr Goldsmith in 2009, only making an exception by inviting him to the royal wedding in 2011.
WATCH: Relive Pippa Middleton's wedding day. Post continues...
Gary also owns a villa in Spain which he has crudely called "La Maison de Bang Bang."
Amid a string of other embarrassing scandals, Gary told the Daily Mail in 2013: "People like to think that I’m this black sheep, a bad boy. I’m not really. I’ve had my moments – I’m not totally innocent – but am I disliked by my family? No, that is simply not true. I’ve got a good heart and I care about people. We have never fallen out."
If you or anyone you know needs support, call 1800-RESPECT on 1800 737 732 at any time.