AI writing computer code may be a norm by 2040 – is the IT industry ready for it?
Researchers predict that, by 2040, machine-generated code (MGC) could become as common as artificial intelligence in today’s devices. The reason for this? Recent advancements in AI generative models led to promising results in automated code generation. We’re not there yet, but we seem to have found a good starting point.
In the following article, we will discuss three MGC projects: Rice University’s Bayou, OpenAI’s Codex, and Facebook’s TransCoder.
1. Bayou
Bayou is a system for generating API-heavy source code in strongly-typed languages such as Java. The model is being trained on openly-available GitHub code, but with a twist. Instead of learning from code per se, the neural generator uses “program sketches,” in which program-specific names and operations have been abstracted out, and only the syntax applicable in a wide range of scenarios remains.
As Bayou creators write in their paper:
Our central idea is to learn not over source code, but over tree-structured syntactic models, or sketches, of programs.
Bayou is designed to be used as follows: the programmer can query the generator for API calls or types they wish to implement. Bayou will then produce relevant code automatically.
Here an example of such code:
2. Codex
Codex is an AI system that translates natural language to computer code.
Codex was created by OpenAI on top of predecessor GPT-3 and, per its website, “has a memory of 14KB for Python code, compared to GPT-3 which has only 4KB — so it [Codex] can take into account over 3x as much contextual information while performing any task.”
Codex was trained on both natural language and tons of open-source code, including GitHub data. According to OpenAI, Codex shines best in Python, but also works with over a dozen other languages such as JavaScript, Go, Perl, PHP, Ruby, Swift, and TypeScript.
In a conversation with Microsoft’s Kevin Scott, OpenAI CEO Sam Altman shed some light on how Codex came to be:
One thing we were curious about is what we could do with code generation. Could we help developers write code? And so we used the Microsoft supercomputer, our generative text models, and we fine-tuned it on thousands of open-sourced GitHub repositories.
Using Codex is as easy as typing natural-language instructions (well, for the most part) and watching the code generator manifest things in the console.
Below is an example of using Codex to create a Space Game in under ten minutes: the user starts with simple instructions like “add the image of a spaceship”, “animate it”, “add the image of an asteroid”, “show ‘Collision!’ each time the two overlap, in red”, etc. – eventually ending up with a full-functioning game.
OpenAI estimates that Codex produces the right code 37% of the time. Let’s see what programmers think of this. Are they scared that AI will take away their jobs at this rate?
A commenter by the name of Rocco Balsamo writes in the comment section of the article that talks about Codex:
I’ve been a programmer for 17+ years, and this does the opposite of scare me – it excites me!
Why? If it works half the time, it will allow me to build more cooler stuff faster. Computers have been moving to higher and higher levels of abstraction: bytecode, assembly language, low level languages, high level languages, low code / no code, AI generated code.
Another commenter, Mel, writes:
… 37% accuracy is horrible. A Roomba vacuum cleaner is 99% effective in avoiding collisions with furniture.
These comments do not change the fact that Codex looks extremely promising. At the same time, its success rate should probably improve for Codex to be really useful to developers, without giving them extra work of another kind. Perhaps this is the promise of the new version of Codex announced in August of 2021 – it is now in private beta.
3. TransCoder
TransCoder is an autonomous transcompiler described in a related paper by an AI research group from Facebook. What it does is it translates source code between C++, Python, and Java programming languages. Trained on open-source GitHub projects, TransCoder demonstrates the best results when translating from Java to C++, with an 80.9% success rate.
Researchers explain that their method relies only on “monolingual source code”, hence no expertise in the source or the target language is required. And that’s one of the reasons their model performs much better than rule-based transcompilation solutions. Here is what the original paper says:
Currently, the majority of transcompilation tools are rule-based; they essentially tokenize the input source code and convert it into an Abstract Syntax Tree (AST) on which they apply handcrafted rewrite rules. Creating them requires a lot of time, and advanced knowledge in both the source and target languages. Moreover, translating from a dynamically-typed language (e.g. Python) to a statically-typed language (e.g. Java) requires to infer the variable types which is difficult (and not always possible) in itself.
The applications of neural machine translation (NMT) to programming languages have been limited so far, mainly because of the lack of parallel resources available in this domain. In this paper, we propose to apply recent approaches in unsupervised machine translation, by leveraging large amount of monolingual source code from GitHub to train a model, TransCoder, to translate between three popular languages: C++, Java and Python.
TransCoder authors believe the technology can help developers migrate between programming languages and integrate services written in different languages much faster and cheaper, significantly aiding modernization of obsolete software.
In conclusion
Nowadays AI outperforms humans on repetitive tasks such as simple coding. Automation is about reducing the time programmers/coders spend on dull, repetitive tasks. This way programmers can be more creative when writing software.
AI taking over humans’ jobs has been debated a lot recently. Still, an opinion that there will appear new jobs in place of those occupied by AI seems to prevail.
Related Blogs
Off-the-shelf AI: adopting much-hyped technology with minimum risk
Read Article
How deep learning helps healthcare and weather forecasting industries achieve more with less
Read Article
