ChatGPT vs. GitHub Copilot: Comparing Generative AI Apps

eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

ChatGPT and GitHub Copilot are both leading generative AI applications, and are similar to one another in important way. Yet there are also crucial differences between them. Which generative AI platform is best for your needs?

• ChatGPT is a popular generative AI solution that can be used to generate basic computer code, along with text, images and other content.
• GitHub Copilot is a cloud-based artificial intelligence tool developed by GitHub and OpenAI specifically to assist users to develop code.

To understand which generative AI solution is best for your purposes, we’ll dig into the similarities and the differences across a range of different criteria.

Also see: Top Generative AI Apps and Tools 

And: Generative AI Companies: Top 12 Leaders

Quick Comparison: ChatGPT vs. GitHub Copilot

ChatGPT vs. GitHub Copilot: Overall

GPT is an abbreviation for Generative Pre-trained Transformer, a form of advanced artificial intelligence (AI). ChatGPT simulates human thought by using a neural network machine learning model trained based upon a vast trove of data gathered from the internet.

The ChatGPT AI-powered language model was developed by OpenAI. It was trained on a massive amount of text data from the internet to be able to generate human-like text responses to a given prompt. It answers questions, can converse with users on a variety of topics, and even generate creative writing pieces. As such, ChatGPT goes far beyond being a chatbot to being able to create documents and articles, and solve problems.

GitHub Copilot is supported by GitHub, whose user base counts more than 100 million developers, with more than 400 million open-source contributions using about 500 languages to build software (JavaScript is the most popular). It has tens of millions of visitors every month from all over the globe. GitHub is a convenient place to store, track and collaborate on software projects while also providing social networking opportunities.

GitHub Copilot is a cloud-based artificial intelligence tool developed by GitHub and OpenAI to assist users of Visual Studio Code, Visual Studio, Neovim, and JetBrains integrated development environments (IDEs). This enables it to write code faster with less work.

Copilot draws context from comments and code to suggest individual lines and whole functions instantly. The OpenAI Codex it uses is a generative pre-trained language model that is trained on natural language text and source code from publicly available sources, including code in public repositories on GitHub.

Winner:

ChatGPT wins on breadth of features and capabilities, while GitHub Copilot wins on the strength and depth of its programming capabilities.

Also see: Generative AI Startups 

ChatGPT vs. GitHub Copilot: Chatbot Functions

ChatGPT is used a lot in chatbot applications to automate customer service, answer FAQs, and engage in conversation with users. It can respond conversationally by tapping into a comprehensive set of online text written by actual people, as well as news items, novels, websites, and more.

ChatGPT does a good job of analyzing information, evaluating online behavior, and even making product recommendations as part of the online sales and upselling process. Automation features extend to appointment scheduling, reservations, payment processing, queries about shipping schedules, order progress, product returns, product and service availability, and more with a high level of accuracy.

ChatGPT can also do image interpretation using multimodal language AI models. This enables it to build websites based on sketches, and suggest recipes based on a photo of what is in the fridge or sitting on a countertop. Further ChatGPT can perform complex tasks. It has achieved some success with basic computer programming duties, but it ventures well beyond that into territory such as drawing up simple lawsuits, creating elementary computer games, passing exams, checking for plagiarism, generating written content, summarizing documentation, highlighting key passages within text and translating into dozens of languages.

GitHub Copilot doesn’t do any of this, but it isn’t designed to. Rather than trying to be everything ChatGPT attempts to be, GitHub Copilot focuses – deeply and effectively – on its role as an AI-assistant for software coding.

Winner: 

ChatGPT wins as a chatbot.

On a related topic: What is Generative AI?

ChatGPT vs. GitHub Copilot: Accuracy of Response

ChatGPT can be prone to error, based on assumptions made on data that may not be current. But most of the time it is accurate.

ChatGPT got into hot water with a few strange responses to queries and a number of completely wrong answers. Fortunately, each new version gets better. GPT-4 added a greater degree of accuracy.

OpenAI stated that GPT-4 is 82% less likely to respond to requests for content that OpenAI does not allow than its predecessor, and 60% less likely to invent answers. But don’t expect it to be perfect. That includes coding. Its programming output should always be verified by human eyes.

GitHub Copilot similarly has some accuracy issues. Users accept on average 26% of all completions shown by GitHub Copilot. In certain languages like Python that goes up to 40%.

In other words, GitHub Copilot does not generate perfect code. Instead, it creates the best code possible given the context it has access to. Thus, code may not always work. At times Copilot may rely on old or deprecated libraries and languages.

Overall, languages like Python, JavaScript, TypeScript, and Go tend to perform better compared to other programming languages. GitHub Copilot is said to increase in accuracy when code is split into smaller functions, when meaningful names are used for function parameters, and good instructions are given. Any code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted. That said, it is better than the code generated by ChatGPT.

Winner: 

GitHub Copilot wins on software coding.

And: ChatGPT4 vs. ChatGPT 

ChatGPT vs. GitHub Copilot: Integration

ChatGPT comes out of the open-source community. It can be plugged into other applications to generate responses via an API. Plugins are becoming available, including those for the likes of Kayak, Expedia, OpenTable, Slack and Shopify with more on the way. It is also integrated with a lot of different programming languages.

GitHub Copilot is well integrated with the Visual Studio Code, Visual Studio, Neovim, and the JetBrains IDEs. It can analyze and pull from billions of lines of code in multiple languages.

Note, though, that it is trained on publicly available code. Brand new libraries, frameworks and APIs are less integrated (and coding accuracy will be lower) as less public code is available for the model to learn from. It takes time for the GitHub Copilot codebase to build up enough examples to provide accurate code.

Winner: 

There is no clear winner when it comes to integration.

For more information, also see: Top Robotics Startup

ChatGPT vs. GitHub Copilot: Security

ChatGPT hasn’t really had much attention on security to date. Developers using it are expected to build in their own security features.

GitHub Copilot benefits from the many security features that have been added to the GitHub platform in recent years. In addition, Copilot for business users provides plenty of coding privacy and other protection measures. An AI-based vulnerability prevention system blocks insecure coding patterns in real-time to make suggestions more secure.

Copilot targets the most common vulnerable coding patterns, including hardcoded credentials, SQL injections and path injections. This helps to detect vulnerable patterns in incomplete fragments of code.

Winner: 

GitHub Copilot wins on security.

On a related topic: Top Natural Language Processing Companies

ChatGPT vs. GitHub Copilot: Pricing

ChatGPT has a basic version available for free but the main ChatGPT Plus version costs roughly $20/month. Those subscribing gain access to ChatGPT at peak times, faster responses, and priority access to new features and improvements.

On top of the basic subscription, there is a pricing scale per 1,000 tokens (chunks of words). 1,000 tokens comes out to about 750 words of material. Costs range from 3 cents to 6 cents per 1,000 tokens for prompts, and another 6 to 12 cents per thousand once finished. The higher rate provides access to a larger set of contextual data.

GitHub Copilot pricing is much simpler. It is $10 a month for individuals and $19 a month per user for business. The business version comes with management, policy and privacy tools for added security and collaboration.

Winner: 

GitHub Copilot wins on pricing.

For more information, also see: Top AI Startups

ChatGPT vs. GitHub Copilot: Bottom Line

ChatGPT uses a transformer-based architecture as part of a neural network that handles sequential data. Although the data it draws from may be a little dated at times, ChatGPT seems to perform OK in coding, and does a very good job on chat, language translation, answering questions, understanding images, and can even determine why a joke is funny.

But ChatGPT is not so much an automatic coding platform as an AI-based system that can be incorporated into development functions.

GitHub Copilot was designed 100% with code generation in mind. As such its output is in code snippets as opposed to the natural language responses of ChatGPT. GitHub is the obvious tool for professional coders as well as for amateurs with plenty of experience.

Winner: 

ChatGPT may be the better option for beginning coders and those who rarely code and aren’t part of the Github community. Copilot definitely aids developers in writing code faster, while ChatGPT can play a role in streamlining the development process but is better as a broader AI system that incorporates chat and some coding.

In short, ChatGPT is more versatile as an overall generative AI to create content of all kinds, including software code. But GitHub is a far more powerful platform for developing software code.

In either case, though, think of them as assistants to developers and content creators and not a replacement. Human minds remain an essential element of solid creation.

Also see: What is Artificial Intelligence?