The realm of Artificial Intelligence is significantly making strides, making life automated and convenient. An AI tool has been developed by Meta, Code Llama, which is designed to generate and elucidate code using natural language, particularly English.
Code Llama is a versatile tool that can both complete unfinished code and troubleshoot errors across various programming languages. From Python to C++, Java to PHP, Typescript to C#, and even Bash, Code Llama has you covered.
Code Llama Versions
Available in multiple versions, including a Python-optimized variant and another finely tuned to understand instructions like “Write me a function that outputs the Fibonacci sequence,” Code Llama stems from the foundation of the Llama 2 text-generating model, recently open sourced by Meta. While Llama 2 had the ability to produce code, it wasn’t necessarily of high quality like others.
Meta trained Code Llama using the same dataset as Llama 2, a blend of publicly available sources from across the web. However, they took things a step further by giving the model extra focus on the training data subset containing code. This allowed Code Llama to extensively grasp the intricate relationships between code and natural language, surpassing the capabilities of Llama 2.
Revolutionizing Code Generation
The codes that Llama generates varying in size from 7 billion parameters to 34 billion parameters, underwent training using a whopping 500 billion tokens of code, accompanied by code-related data. To further refine the Python-specific Code Llama, it was fine-tuned using an additional 100 billion tokens of Python code.
Similarly, the instruction-comprehending Code Llama was honed with the help of feedback from human annotators, ensuring it generates responses that are both safe and helpful for questions posed to it.
Among the array of Code Llama models, many are equipped to seamlessly insert code into existing code. All models can process approximately 100,000 tokens of code as input, while the 7 billion parameter model has the capability to run on a single GPU.
Notably, Meta asserts that the 34 billion-parameter model is a standout performer in the realm of open-source code generators, boasting the highest performance to date and claiming the title of the largest model in terms of parameter count.
