AI in Software Development: Ready for the Pilot Seat?

August 22, 2024
Written by
Reviewed by
Diane Phan
Twilion

This is the written form of my 2024 conference/meetup talk that I presented, among other places, in Magdeburg, Bucharest, Nuremberg, and Munich.

AI in Software Development: Ready for the Pilot Seat?

The hype around artificial intelligence has repeatedly broken all records and raised enormous expectations. Stock of chip manufacturers and other AI companies skyrocketed, and the capital market has been extremely bullish.

A lot has happened in the AI ecosystem since ChatGPT wowed the masses two years ago. AI Chatbots and text-to-image models, such as DALL-E, became helpful tools for finding information or assisting with creative tasks. These tools can create seemingly reasonable content within seconds that would otherwise require hours or days from humans. Sure, there is no tool that can write a viral blog post end-to-end, but you can use AI to draft an initial post and then refine it. Perhaps even this post was initially drafted using AI to overcome the infamous Blank Page Problem.

AI in software development

AI has changed how I work creatively on my job as a software developer.

The most defining characteristic of Generative AI is its ability to learn from patterns and generate new content. Software code, with its well-defined syntax, has plenty of patterns. GitHub Copilot acts as a helpful co-programmer, providing code suggestions during the coding stage, where it provides code suggestions as developers type. Today, I can't imagine going back to a world where I have to write software without these smart autocomplete suggestions. However, coding currently accounts for only around 10% of a developer’s time, so optimization potential there may seem limited. At the same time, the question of what tasks occupy the remaining 90% of their time arises. And how can AI be used to make developers more efficient during this time?

As a developer, I was eager to explore the AI tools available today to see how they can accelerate the development process. So, I've spent the last few months digging into the realm of AI tools for developers. I have tested, fiddled with, and tweaked a variety of these tools. Today, I'll share some highlights I encountered.

AI as a UX Designer: From text prompt to UI marvel

A UX designer is responsible for creating seamless and engaging user experiences by deeply understanding user needs, behaviors, and pain points. They translate this understanding into intuitive interfaces that are easy to navigate and meet both user and business goals. While a creative UX is important to the core functionality of the product, there are many aspects of a software solution where a designer can rely on best practices, such as the login screen or the user settings.

AI can accelerate these common UX tasks by transforming text prompts into UI designs, outputting assets such as PNGs or Figma files instantaneously. This innovation drastically cuts down on the time-consuming aspects of initial design, enabling designers to rapidly prototype and iterate. Consequently, designers can devote more time to refining user interactions and addressing nuanced design challenges.

Two examples of this category that I found during my research are Rendition Create and Galileo AI. Both tools were able to create intriguing mockups from user inputs in an intuitive manner. They present users with various color schemes for the created design, enable iterative adjustments, and allow for various export formats for the created mockups. This allows human UX designers to pick up the work where the AI stopped.

An AI UX designer suggesting two visual mockups based on a text prompt

These tools work great if you build new software, but it’s challenging to add the context of existing legacy software that needs to be enriched with user interfaces that fit in seamlessly. While these tools generate beautiful designs, I did notice that they often do not fully cover the intended scenario and required manual tweaking. This limitation reiterates the need for the human creativity of designers to achieve unique, trendsetting designs. This workflow is most efficient especially for smaller teams that otherwise wouldn’t have access to a UX designer.

You might argue that these tools don’t really focus on traditional software development tasks, and they don’t appear to fall within a software developer's skill set. Fair enough, but I’m sure the next set of tools does.

AI as a Frontend Developer: From prompt to UI components

A frontend developer is responsible for translating design mockups and user requirements into interactive interfaces using technologies such as HTML, CSS, and JavaScript. This role is crucial for creating a seamless and responsive user experience, but at the same time, frontend developers typically spend a lot of time implementing new user interfaces, which can be very time-consuming and include a lot of boilerplate code.

AI tools can accelerate frontend development by converting text or image prompts into reusable UI components within seconds. This eliminates the need for developers to manually implement designs from scratch or constantly refer to design system documentation, thereby accelerating the development process. The AI-generated components can then be imported into codebases and fine-tuned, allowing developers to quickly iterate and focus on more complex problem-solving and feature development tasks.

Two examples of this category are v0 and Magic Patterns. Arguably, the most exciting feature of these tools is their capability to incorporate images along with text prompts. This enables developers to avoid writing long prompts that describe an interface and rather attach a whiteboard sketch or screenshot from an existing UI. After all, an image says more than a thousand words.

Depending on the service, 3–4 sample user interfaces are generated from the initial prompt. It’s unlikely that one of these suggestions already satisfies the real-world problem. That’s why the tools allow the developers to pinpoint an arbitrary area of the design and add another prompt for refinement. For example., the user can select the space between a submit button on a form and specify “Add a red reset button that has rounded corners above the blue submit button”.

Once the result is satisfied, users can export the design to Figma or download the code for the component. If you decide to export the component as code, you’ll see that it is implemented as a React component with Tailwind CSS, a popular CSS framework, and an open-source icon library. For projects that already come with their own design, the tools also offer to bring your own themes or even design systems to match the look and feel of existing user interfaces.

 

 

However, these tools also have a few limitations. Although they do a reasonable job at producing the components described, their accuracy heavily depends on the specificity of the user prompts. It is an innovative concept, but its performance fluctuates between highly satisfactory results and missing the mark on more complex or unique component requests.

The tool is effective, but it does have its blind spots. Especially when one tries to describe more complex interfaces.

The AI has been prompted to build a simple form-like page and was able to render all relevant boxes and suggest reasonable values as options for the select input.

These are some tools that made the lives of frontend developers easier. But how about backend devs?

AI as a Full-Stack Developer: From concept to completion, in a jiffy

A full-stack developer handles both frontend and backend development, creating comprehensive applications by managing multiple disciplines such as designing server applications, database schemas, and client-facing interfaces. Important considerations include scalability, security, performance, and maintaining clean and efficient code across both ends of the stack.

AI has the potential to accelerate full-stack development by automating the generation of boilerplate code for data structure definitions, HTTP request routing, and CRUD (Create, Read, Update, Delete) operations. This significantly reduces the time spent on repetitive tasks, allowing developers to focus on advanced functionalities, system architecture, and problem-solving. By relieving developers of these foundational tasks, AI-enhanced tools enable quicker prototyping and faster deployment of features, ultimately increasing productivity and code quality.

The AI tool suggested multiple reasonable user stories based on my initial prompt

There are two tools that can help with full-stack development of projects that start from scratch: Marblism and Glowbom.

I found Marblism intriguing. After giving the AI a prompt about creating a web-based email client, it took me through a systematic, wizard-like process, guiding me from creating user stories, defining the database model, to the eventual generation of a full-stack application. This NextJS app could then be downloaded or deployed directly on their platform.

Beginning from user stories based on your text prompts to creating the data model, establishing pages that cater to specified tasks, all the way to deploying the application, the AI covers an extensive part of the backlog.

It’s important to keep in mind for these full-stack app generators that they require precise instructions and constant supervision to churn out valuable results. While I found the resulting full-stack app very impressive, the UI does not have the same UX as the ones created with the other solutions on top.

Another limitation is that the entire app is generated all at once as a full application. Hence, it’s difficult to go back later to make smaller improvements. However, these tools can support citizen developers in building apps without having to write code.

All of these tools have produced content that lives independently of an existing codebase and requires manual integration. The next tool will be able to understand and build on your existing repos.

AI as a Junior Developer: Fixing bugs or adding simple features

In the early stages of a developer's career, typical tasks often involve getting familiar with the codebase, working on bug fixes, and making small, incremental changes. These tasks are usually straightforward, focusing on understanding the existing logic, replicating patterns, and ensuring consistency across the project. While these contributions are vital, they are usually made under the close supervision of more experienced developers, who review the code to maintain quality and coherence within the broader application.

Given the repetitive and often well-defined nature of these tasks, AI tools can serve as a valuable asset for automating parts of the process. Many of the fixes or features junior developers handle are derivable from existing code patterns within the codebase, making them prime candidates for AI assistance.

The AI tool showing the specs it derived from the reported issue. It also shows the next step in the process to fix the problem

One of the most promising tools in this space is GitHub Copilot Workspace. This tool takes AI-powered development a step further by integrating directly with your repositories on GitHub. It allows developers to issue text prompts or use existing GitHub issues as the basis for code suggestions. As it processes the prompt, Copilot Workspace guides the developer through its reasoning, offering suggestions and potential solutions along the way.

Once a developer agrees with the AI’s suggestions, the proposed changes can be automatically converted into a pull request. This pull request can then run through the project's CI/CD pipeline, ensuring that it meets all the necessary quality checks before being approved and merged.

GitHub Copilot Workspace is still in technical preview and waitlisted, but it already shows great potential. I noticed that it was able to solve smaller UI fixes reliably, but struggled with more complex tasks (for example, the import and usage of new dependencies that involved too many lines of code). However, this might also be where Junior Developers would struggle initially.

More tools to discover

The list of AI dev tools goes on. It also contains helpers that can automate various development tasks. For instance, PR review agents can automatically review and comment on pull requests, identifying issues and suggesting improvements.

There are other tools that can support you in writing documentation from code, ensuring that it remains up-to-date with minimal effort. Shell assistants can automate command-line operations, making complex tasks easier to execute. Migration assistants help with updating and transitioning codebases to new frameworks or versions seamlessly. Test writers can automatically generate unit and integration tests, ensuring code quality and reducing the manual effort involved in testing.

With AI dev tools bolstering productivity, developers will have more bandwidth to focus on the more creative, strategic, complex, and value-adding aspects of their work.

GitHub Repository with a list of awesome AI dev tools

It's evident that there’s room for further developments and enhancements to these tools. In the meantime, they can save you plenty of time especially when you’re starting from scratch,d building prototypes, and proof of concepts. These tools cater to an array of development-related tasks - making the development process more efficient and manageable.

If you're intrigued by AI tools that simplify the development process and reduce the complexity of building autonomous agents, Twilio AI Assistants might be just what you need. Instead of wrestling with memory management for AI conversation context, over-engineering complex Retrieval-Augmented Generation pipelines, or integrating various systems and channels, Twilio AI Assistants handle the heavy lifting for you. They even help with measuring LLM hallucinations and detecting prompt injections, allowing you to focus on what truly matters: delivering an exceptional customer experience.

 

What does the future of AI hold for developers?

While the introduced AI dev tools cannot replace the critical thinking, creative problem-solving, and strategic understanding of experienced developers (yet), they offer notable support that makes the day-to-day job more productive. These tools help developers free their mental capacity to focus on different aspects of software design, and they are here to stay.

Today, a developer might use such tools for repetitive and time-consuming tasks, but tomorrow's developer might useAI to generate entire applications or resolve issues in a decade-old legacy codebase. This, in turn, may change the skills developers need to emphasize, focusing more on problem-solving, improved user understanding, and communication skills.

No matter how it evolves, one thing is sure: it's an exciting time to be a developer in the age of AI, and these tools offer significant potential for growth and innovation. Are you also excited to build with AI?

We can’t wait to see what you build!