AI assisted CAD

A CAD Copilot

I was working with CAD now for quite some years and I am always surprised of how much manual and repeating steps still need to be done. A lot of the work when one designs for example connecting plates between two parts can be simply derived automatically by the distances observed in the two parts that should be connected. In the end it is a bit like the boilerplate one needs to program while writing scientific software.

Having an AI for CAD that is similar to GitHub Copilot would be amazing. It could analyze the existing object tree or sketch and provides suggestions for the predicted next steps. Thereby, differently to most existing solutions it should not predict the final model but rather the usable work steps so that they can later be modified manually. When using a software such as solvespace, where the model has a very simple textual representation, most of the existing large language models should be capable of doing the job. Not sure how models of e.g. solidworks are stored but I am sure there is a similar concept behind.

The preview of the next steps could be visualized in the main viewer as some semi-transparent structure around the existing tools and a keyboard shortcut could be bound to toggle between possible solutions as well as accepting them for the next step.

The impact is huge, looking on the amount of people that use CAD on a daily basis ranging from city planning tools, mechanical and civil engineering, product designers, architects or hobby 3D printing enthusiasts.

Feedback from other automated tools

There is other tools around such as finite element modelling (FEM) that could guide the AI to reasonable results or provide some feedback to it while it is proposing the next steps (some sort of supervised learning). In the end one could even imagine that for example an entire sketch of a family house could be done by an AI. You provide the input of the size of the plot, the number of rooms and levels, the number of bathrooms wished for etc. and the tool provides a fully functional and modifiable CAD model of the house. That initial model was already tested through a short FEM to make sure the solution is feasible.

While this is of course not a solution to generate a final product, it might be a way to sketch first ideas quickly with the potential house owners and for example create three versions of the house where they can then decide which plan should be brought to the next planning stage. This avoids time consuming manual readjustments after creating the initial sketch.

Challenges

There is a number of challenges that sadly render this problem too complex to be solved on a small scale. Those are probably also the reasons why none of the big companies have delivered anything usable despite probably most likely invests tens of millions into this every year.

Training datasets

Besides the software stack, one would of course also need some large training dataset for the AI that has high quality. While this is available through open source code for the programming domain, it is really not available for CAD. All one can find is pre-exported 3D models in stl or step format on maker forums for mechanical parts (e.g. thingiverse). But those miss the essential working steps that would be required to generate the workflow suggestions.

Complexity of software

While a code suggestion AI simply needs to be trained on text and the output can be shown in a text editor as a simple in line suggestion, the CAD preview is much more complex then this and requires complex UX and UI design elements to work.

Industry has no standard

While the industry has standards for output formats such as step, there is no such thing as an exchange format of functional modifiable objects. That would be the requirement to build a toolchain that can couple into different backends. Also most big companies heavily rely on their existing models which are usually several thousands of objects and files that would be impossible to port to any new format.