This seems to be a really common question now and the answer is not that simple. We know this question is on the minds of business owners, management teams, and coders alike so we set out to get some answers.

The Ground Rules

It wouldn’t be fair to answer this question without first establishing a few ground rules. You may say “Well that’s not fair!” or “This AI can do that better” or some other idea. These are all fair comments. In addition, AI is improving every day and new tools are becoming available every day so we will update some content on this page over time but the test will likely remain static but we may create a part 2 of this same article before too long.

Here is what we set out to do:

• Create a hockey in/out app for games with a few extras
• Have an idea that was decent but was not fully thought through
• Add features as we went as new ideas emerged
• Keep the coding simple to limit the use of frameworks
• Simply update files with no need for compiling, etc.
• Leverage some of our server resources (PHP 8.x, Apache, MariaDB)

There are also a few assumptions here that we should consider. These are relatively important to give some context around the why and the how.

• The person prompting AI is tech savvy but not a current developer
• The person prompting AI understands PHP and is comfortable debugging issues
• We have access to the servers via SSH or other methods to update files and monitor logs, etc.
• Use ChatGPT or Grok as the AI web app developer

Alright, the fun begins.

The First AI Prompt

After a little bit of planning and working out the idea, we took the entire idea and wrote it down. It was a few hundred words and we thought fairly well described the idea along with some basic requirements around the technology…at least we thought. Here is where we made our first decision. We looked at what both AI options gave us and decided to move forward with Grok. At the time of this writing, we built this web app using Grok 3.

It’s important to note that this article is not a comparison of AI models. We had to pick one model to move forward with and Grok 3 had the better response at the time of this writing.

Right out of the gate we ran into a bump in the road; we forgot to tell the AI not to use a PHP Framework, so it spit out an idea that was going to require using some composer packages which, under normal circumstances is great, however, it’s not what we were looking for. With another quick prompt we were able to have Grok output another structure and some basic files.

We will say, the structure of the files that Grok created was very interesting. We could have certainly prompted the AI to change this in some way, and later on we do make some changes, however, overall this was quite good for a basic web app.

The Progression and Challenges

We will spare you some of the boring details, however here are a few notes about the process that we found the most interesting along the way.

App Look And Feel

Our first discovery was the look and feel of the AI generated app… it was ugly! We were smart enough to ask it to take advantage of Bootstrap and Font Awesome so we didn’t have to code all of the CSS on our own, but beauty was certainly in the eye of the beholder. Quite a bit of time was spent making adjustments to the look and feel of the app; prompting the AI for improvements, feeding it ideas, updating the code, and testing the changes.

The Prompt Speed

At first, Grok was able to take in your input quickly and respond in a fairly timely manner, however this changes dramatically once you have prompted the AI 50+ times and had to make a lot of code changes.

If you can imagine, over time this app consists of well over 30 files, and in some cases the changes needed to be applied to 10 or so files if our request to Grok was fairly complex. Over the course of a few days, we found the browser tab to be slower than expected. Even trying to type into the AI prompt would result in delays or pauses for 5 seconds or longer. Once this starts happening there isn’t much you can do, as Grok, in our example, can’t reference past chats very well, so starting a new chat would possibly create more problems than solutions.

Forgetting What Was Done (limited memory)

Strangely, every now and then, the AI would forget what was previously done. This situation was not frequent, however keen eyes were required to trap this issue. The results of AI memory loss were pretty damaging. In several cases, replacing code removed previously completed functions, thus destroying some functionality. In other cases we would have duplicated code, making it harder to solve future issues at a minimum, but even worse, having multiple methods for doing the same thing.

A good example of this was when, early on, we asked AI to leverage SendGrid for sending email from the web app. We used their free API, and with a little bit of elbow grease, we were able to get it working. Later in the process we needed another email that was going to be sent from the app. When adding this new email feature, AI created an entirely new email routine for sending direct through Sendmail on the server. With a little careful prompting we were able to remind AI of this and prompt for some improved code centralization which was handled quite gracefully.

Breaking Code

AI is far from perfect. We can’t tell you how many times we had to help correct it. A very common response from AI was “Oh, you’re right!”. We were happy to see that it could handle the feedback, however changing 5 or more files and then seeing that the added code was completely broken was a little frustrating. In some cases we were able to identify issues in the new code, however in other cases, there was so much broken code that it was easier to revert back.

This is where a novice developer or non-developer would get into some deep water and likely drown.

Leveraging the error log on the web server and the browser console was a must. Taking the code verbatim, while in many cases worked, was not always the answer. As the files got larger, AI would only give snippets of code to replace or add to sections, so careful surgical insertions were required.

Bad Code

We would like to say that AI created great code however that is simply not the case. Having said that, humans don’t always create great code either. In most cases we found that the code worked, however in many cases we found that the code either strayed from the pattern previously established, was inefficient, or had other issues. A quick example of that is when we prompted AI to use Bootstrap. It decided to use an older version instead of the latest version. While we are skilled and experienced enough to catch this, someone who did not have the skills or knowledge necessary to identify and correct it would have simply moved forward with an older framework for no reason, not to mention creating a potential security risk.

Lessons Learned

There were many lessons learned throughout this process, however the most valuable one was likely to approach it with better up-front planning. This is something we do as a company when building a web or mobile application, so this should be done at the start of this process as well. Making changes part way through creates quite a few challenges, in addition to being very time consuming, so the more that can be figured out up front the better.

Most of the other lessons are around the limitations and challenges listed above. Expecting these situations and knowing how to solve them is super important. Understanding that creating an app with AI has a short shelf life unless it is managed by a highly skilled development team. This was a big lesson as well.

Can I See the Web App?

Yes! You can register for free and give it a try to see what was built. We even used AI to create a logo and build a simple website. The challenges with both of these processes would require an entirely different article, however you can see it all here: https://lockerlineup.com

Conclusion

Overall we were quite impressed with AI capabilities in creating a web app. The quick answer to whether AI can create a web app or not is “sort of”. AI certainly can’t or won’t create the entire app from the ground up to completion. For this, a developer needs to be involved, and a lot of planning needs to take place.

What about an MVP (Minimum Viable Product) version? Again, this is about the same answer as above. I would say that the Locker Lineup app is exactly that, an MVP. As it gains popularity it will need a complete rebuild from the ground up with a solid framework, more enterprise security, and with scalability in mind.

While AI was responsible for much of the code, a previous developer with technical training was involved from the start to make adjustments and debug issues. If this was not the case and someone with little to no development skills were to attempt building an app with AI, this fairly small app and work effort would have most certainly failed or would have taken substantially longer to complete (along with a lot of frustration!).

If you’re in the market for a custom app, Oracast is here to help you, starting with the first and most important step in the process: planning a web application. Our Phase 0 approach ensures you have a well documented app before any lines of code are written. This is critical to the success of an app and allows for a more accurate estimate of effort.

Oracast Staff

• Web Apps