Blog: Using ChatGPT for programming

For this blog, I’ve collected our developers’ experiences with using machine learning tools for programming. The results can be summarized with the following three questions:  

For which applications is machine learning useful?

• Summarizing text. If there is little time and a lot to read, machine learning is a good helper for going through long texts. By extracting the essential things, machine learning tools save the user’s reading time. Of course, machine learning is not perfect and it may miss some essential things from the text (though to be fair, a human might too). That’s why it’s essential to be clear about what details you want to summarize from longer texts. For example, when a developer receives new code without documentation, machine learning tools can tell them in natural language what the code does. 
  
  
• Creating documentation. Machine learning also works as an assistant for creating documentation – especially when the developer who’s been in a project for a long time may no longer see the big picture. Machine learning tools can create checklists of key points that should be included in the documentation.
  
  
• Generating text. Machine learning is good at generating content by giving instructions on what the writing should be like. For example, if you need versatile test data for code testing, machine learning tools will create that based on your requests. Doing this by hand would be a very mechanical and time-consuming process. As another example, machine learning can make use of the documentation to create the necessary parameters for the code for the scenario in question.
  
  
• Extracting negative and positive words. Machine learning tools can extract negative and positive words from natural text. With this information, it can determine which things delight users and which things anger them by identifying the emotional tone behind a body of text. In other words, machine learning can be used to perform sentiment analysis (so-called opinion mining). As one practical example of this, large amounts of feedback text can be run through a machine learning tool to summarize what’s working well and what needs fixing. This saves considerable time when going through customer feedback.
  
  
• Creating software code (in some cases). Machine learning tools can create software code from natural language such as retail software. Machine learning tools don’t write perfect code, but they can be helpful – especially at the beginning of a project when a single line of code hasn’t yet been written. You can ask the tool to write code suitable for the situation. Although the code in question likely cannot be used as such, with editing it may become usable.
  
  

What adverse effects can come from the use of machine learning?

There are many potential pitfalls to using machine learning tools for generating code. Machine learning does not necessarily create the kind of code that the developer originally intended. Programming skills are still needed to correct programming errors. The code may work on the surface, but there may be surprises under the hood. Code generated by machine learning tools may also be less efficient and run slower.

The code generated might also have licensing or some other legal issue if it’s derived from some unverified or unlicensed source. If possible, one should verify the sources being used to generate the code in question.
 
Entering sensitive code into a third-party system is not a good idea for information security reasons. The information to be entered may require anonymization before being processed by machine learning tools, which again takes time, raising the threshold for using it.

What are the shortcomings of machine learning?

At the moment, machine learning tools like ChatGPT have to be given a lot of guidance in order to get it to take the necessary actions. This often takes many attempts to get things moving in the right direction, which takes time. It’s not necessarily worth using these tools for complex situations because it may take so much time to give the necessary instructions – meaning you could do the work yourself in less time. Certain AI tools like ChatGPT also have a limit of 4000 tokens. This is the tool’s working memory and when it fills up, the instructions given to it start to drip out of use.
 
Even with these drawbacks, machine learning is a promising tool for programming. It can be compared to a dishwasher – it does the mechanical work well enough. Of course, a person could do the work in question better, but using it frees up time for other more creative tasks that are more meaningful. With that in mind, now is a good time to try out and explore new ways of programming using machine learning as a tool to help the process.