If you are familiar with natural language processing (NLP), you might have heard of GPT-3, the third iteration of OpenAI's large language model systems that can generate coherent and diverse texts on almost any topic. But did you know that there is a newer and more advanced version of GPT that was released earlier this year? Meet GPT-4, the latest milestone in OpenAI's effort to scale up deep learning and create increasingly sophisticated and capable language models.
What is GPT-4?
GPT-4 stands for Generative Pre-trained Transformer 4, a multimodal large language model that can accept both image and text inputs and produce text outputs. It is trained on a massive amount of data from the internet, such as books, articles, social media posts, images, etc., using a technique called self-attention that allows it to learn patterns and relationships between words and concepts.
GPT-4 is based on the same architecture as its predecessors, but with several improvements and innovations. It has more than 100 billion parameters (the basic units of computation in neural networks), making it 10 times larger than GPT-3.5 (the previous version) and one of the largest models ever created. It also leverages more data (about 1 trillion words) and more computation (using a custom-built supercomputer co-designed with Azure) to achieve better performance and accuracy.
What can GPT-4 do?
GPT-4 can do many things that previous models could not or could only do poorly. For example, it can:
- Solve difficult problems with greater accuracy, thanks to its broader general knowledge and problem-solving abilities. For instance, it can pass a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5's score was around the bottom 10%.
- Generate, edit,vand iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays,or learning a user's writing style.For example, it can explain the plot of Cinderella in a sentence where each word has to begin with the next letter in the alphabet from A to Z without repeating any letters: "A beautiful Cinderella dwelling eagerly finally gains happiness inspiring jealous kin love magically nurtures opulent prince quietly rescues slipper triumphs uniting very wondrously xenial youth zealously."
- Handle much more nuanced instructions than GPT-3.5 by understanding context better. For example, it can find common availability for a meeting given different schedules: "Andrew: 11 am - 3 pm Joanne: 12 pm - 2 pm Hannah: noon - 12:30 pm Common availability for a 30-minute meeting: noon - 12:30 pm"
How safe is GPT-4?
One of the main challenges of creating large language models like GPT is ensuring their safety and alignment with human values. OpenAI has spent six months making GPT-4 safer and more aligned, resulting in a system that is 82% less likely to respond to requests for disallowed content (such as harmful or offensive content) and 40% more likely to produce factual responses than GPT-3.5 on internal evaluations.
To achieve this, OpenAI has incorporated more human feedback, including feedback submitted by ChatGPT users, to improve GPT-4’s behavior. They also worked with over 50 experts for early feedback in domains including AI safety and security². Moreover, they have applied lessons from real-world use of their previous models into GPT-4’s safety research and monitoring system. Like ChatGPT, they will be updating and improving GPT-4 at a regular cadence as more people use it.
Posts
0 Comments