We analyze Google's Gemini launch marketing and question its authenticity.
It's December 6, and Google just released a significant amount of marketing content for their new model, Gemini. There were over 15 different videos showcasing the capabilities of this highly anticipated model. Let's take a closer look at the main demo video and analyze its authenticity.
In the main demo video, Gemini appears to respond to audio and video cues. However, a blog post from Google for Developers reveals that the model actually responds to text prompts and images, with the rest being video production. This raises questions about the truthfulness of the presentation.
Some audio prompts in the video are oversimplified compared to the actual text prompts used. For instance, a prompt involving a drawing and the order of planets is more complex in reality, requiring Chain of Thought reasoning. This simplification can be seen as another instance of truth bending.
Another segment of the video shows Gemini identifying a sleight of hand trick. However, the blog post indicates that extensive prompting was required to achieve this understanding. This again suggests that Google edited out much of the prompt engineering necessary for the model to respond effectively.
Google claims that Gemini beats GPT-4 in various benchmarks. However, these claims mainly apply to Gemini Ultra, which won't be released for several months. The benchmarks show that Gemini outperforms GPT-4 using Chain of Thought reasoning over multiple samples, but this technique seems forced and unconfirmed by the community due to the model's closed source nature.
While Gemini Pro has been deployed to Bard, Google's equivalent of ChatGPT, it offers performance comparable to GPT-3.5, a year later than OpenAI's release. The full capabilities of Gemini Ultra, which allegedly surpasses GPT-4, are yet to be seen.
The marketing for Gemini appears to be more hype than substance at this point. Without open sourcing the model or providing more straightforward demonstrations, Google's claims remain questionable. I suggest that Google should return with their marketing once Gemini Ultra is released and, ideally, open sourced.
