V2 Brainstorming

[rianadon]

After creating this tool and building my first Manuform I've come up with a lot of ideas of what could be done better. Below is a list of what I'm thinking in case anyone has feedback.

Model Generation

1. Use OpenCascade instead of OpenSCAD. While both are pretty mature, OpenCascade can export STEP files (big win for dropping models into < Your favorite CAD package > and modifying the keyboard) and can model with splines (enables smoother-looking keyboards). And an added bonus is that the models do not have holes!
2. Use replicad for modeling. I've already shot myself in the foot trying to use OpenCascade directly, and there's no good reason for me to continue doing so. Replicad has great docs & the api has TypeScript types, which makes developing very easy. However, Replicad is lacking many features of OpenCascade, so I will have to fall back on the OpenCascade primitives for advanced features.
3. Develop in TypeScript. While Clojure's functional programming style works great with this application of building up a 3D model, Typescript autocomplete and its extremely fast compilation (especially with Vite/esbuild) make it a more ideal language. Plus, it will work great with Replicad.

Develop a New Design from (Almost) Scratch

I tried porting the rendering code to use OpenCascade, but it takes an extremely long time to generate (i.e. too long for fast previews), and operations like hulls just don't make a lot of sense in OpenCascade's BREPs (they make more sense for a CSG-based library).
If I'm going to use OpenCascade, I need to rewrite the model to be efficient to generate. Here's how I'm thinking of doing this:

1. Work with points first. To generate the plates, the current Clojure/SCAD code first generates the wall assembly (which involves drawing skinny cubes at the keypoints of the case boundary taking a few convex hulls) then projects the assembly to the XY Plane. While this fits the geometry-heavy paradigm of OpenSCAD, this approach is not very flexible (V1 is unable to fill in the plate) and leads to time wasted generating geometry that is purely for computation and not modeling. Instead, I should first start by finding the 3D location of every key. Then the 3D location of every keypoint on the wall. Then I take all these points and start drawing geometry whether it be for the walls or the plate...

2. Eliminate magic. A large chunk of the V1 code is devoted to drawing the webs and the case because the geometry of these parts are all hardcoded for every configuration of thumb cluster. This code is difficult to both interpret and maintain, and it is also inflexible: I'd have to write even more code to accommodate new thumb cluster designs. Instead, the locations of the walls and webs should be computationally generated based on the key locations (which need not even be in a typical manuform shape). I can use triangularization meshing algorithms to create the web.

3. Caching. Currently operations like generating the hotswap holder are repeated for every single key, which takes up a lot of time. This single model could be cached and copied.

4. Pre-generation. Models like the hotswap holder are not dependent on any user configuration, so they could be generated in the process of building the website then imported. This will especially help with larger models like trackball holders.

5. Generate in parallel. Assuming generating the keypoints is fast, every component of the case (key holes, web, case, bottom plate) should be able to be generated in parallel.

6. Show intermediate results. Generating items like the keyholes should be quite fast, so they can be shown as the case generates to give faster feedback. While this gives results faster, psychologically the intermediate results make the generation step appear to take longer.

7. CSG as a backup? If these optimizations are unable to bring rendering to a speed fast enough for fast previews, I could use a library like Manifold for CSG that will hopefully not create lots of holes in the model.

Personalization to Hand Shapes

The Manuform is meant to be customizable to the user's hand shape, but it's very intimidating starting at a bunch of offsets and tenting angles and being expected to know what's right for you.

1. Use Hand Pose Detection to build a hand model. The TensorFlow hand model seems to be promising in creating a model of a hand from video. Asking the user to position their hands in a comfortable position, I could analyze what a good home row layout might be and use the hand model to figure out tenting from there.

2. Inputs relative to hand. Rather than asking "how many mm left should the H key be next to the J key", I should ask "what angle do you wish to move your wrist and index finger to press the H key?"

3. Come up with a new name. Even though this model will share many similarities with the Manuform, it's shape & tenting may end up looking different depending on how it's adapted to your hand. It should have a unique name to distinguish itself from the Dactyl Manuform.

User Experience

1. Multiple Pages: Measuring hands, editing shaping, and configuring keyboard connectors/keycaps are separate processes. They can be split into multiple sections/pages/tabs if there's too much for one page.

2. Generate Diagrams: Wiring diagrams and perhaps even generate flat PCBs that can be wired together.

3. Ergogen-inspired editor: If the code is flexible enough to create a keyboard out of any key layout, power users should be able to create any key layout they like. I'm thinking of a tool like ergogen.

[fre-sch]

Model Generation 1, 2, 3:
These seem like you should follow your personal preference, it'll be more satisfactory to develop in an environment and using libraries you actually are comfortable with. That being said, it sounds very reasonable to proceed this way.

Develop New Design

1. "Work with points first."
   That's the approach I would choose, it's similar to my own approach in CAD (Fusion 360) when creating the main cluster. As for code design, I would create a grid object type to define base point, offset vector, normal. Add a specs field for switch type, cap size, this would also allow non-key grid object types (wheels, trackpoints, etc). Additionally the grid objects could have an active field for those odd incomplete rows and columns seen in Dactyl Manuform and others, so some select "keys" can be excluded.
2. "Eliminate magic. Instead generate meshes for features."
   Sounds reasonable and desirable.
3. "Caching."
   I think that's more commonly called Instancing in the context of 3D geometry? Sounds reasonable and desirable.
4. "Pre-generation. Models like the hotswap holder."
   Sounds reasonable and desirable.
5. "Generate in parallel."
   Sounds reasonable and desirable.
6. "Show intermediate results."
   It could be fun and somewhat helpful while developing a keyboard to see these intermediate results, but I'd consider that a lower priority nice-to-have. How much value does something like that provide compared to the effort to implement it? How much fun is it for you to implement that?

Personalization to Hand Shapes

1. "Use Hand Pose Detection to build a hand model. The TensorFlow hand model"
   Not everyone has a camera, and I'd say there's significant spread in the performance of webcams (and user room lighting) possibly causing issues for such a feature. Integrating and processing camera input in browser sounds to me like it'll have performance issues. Additionally, AI NNs don't always create desirable results, and might just require a lot of tweaking of the result (by users). It also seems like a feature that could be built later on top of a good parametric base system. Definitely interesting, and sounds like fun to pursue as additional feature.
2. "Inputs relative to hand."
   Sounds a bit like the problem Inverse Kinematics solve: Given a multi-segmented structure it's difficult to specify the individual component orientations to reach a target. Specifying the target location instead, and then solving the multi-segment orientations is more intuitive. In the terms of a keyboard, it might just be easier and more intuitive to position the keys, letting the wrist and fingers solve naturally for the multi-segment orientations. Having to specify multiple relative position/rotation constraints per key sounds exhausting. Also it might not even be possible to solve for such a matrix of constraints.
3. "Come up with a new name."
   Agreed. For one it'll become your own project, and for the other if you're fundamentally rewriting the layout, generation, rendering of the keyboard, you should give it a name that reflects that. Especially as all these plans of yours would lead to something like Dactyl Manuform being only one of many layout outputs.

User experience

1. "Multiple Pages."
   Sounds reasonable and desirable.
2. "Generate Diagrams"
   Sounds reasonable and desirable. PCBs are probably tricky, given how three dimensional the output could be. A general wiring diagram however could be used to design the PCBs in other software. I can't speak to much on this, since I'm fine with 3D printing the case and hand-wiring it, as it cuts out the delay and cost of getting PCBs made.
3. "Ergogen-inspired editor"
   I would personally start with "code" (or YAML like ergogen) as main interface. This would also immediately solve the export/import/sharing of configuration, not that that's really difficult given modern browser APIs. Using that code-centric approach could also help iterate the base functionality of designing a configuration model, generating geometry from that and rendering it. There would also be less churn designing an alternate UI once the configuration model has stabilized. This could also lead to a code design where configuration model is decoupled enough from generator and renderer to allow development of alternate UIs with other frameworks.

[rianadon]

Wow thank you for all the advice! This is more than helpful.

New Design

Add a specs field for switch type, cap size, this would also allow non-key grid object types (wheels, trackpoints, etc). Additionally the grid objects could have an active field for those odd incomplete rows and columns seen in Dactyl Manuform and others, so some select "keys" can be excluded.

Thank you! I almost forgot to add a field for the key type. Here's what I have so far:

offset: Transformation
aspect: number // For accommodating 1.5u keys, etc
type: "mx" | "mxSnapIn" | "alps" | "choc" | "blank" |
      "mx-hotswap" | "mxSnapIn-hotswap" | "choc-hotswap" | "blank"
// Plus row, column, beta, alpha for non-thumb keys

"Show intermediate results."
How much value does something like that provide compared to the effort to implement it? How much fun is it for you to implement that?

I already implemented it, but with model generation has been so far taking about 2 seconds on average on my Macbook which is comparable to what it is now. That's low enough that hiding the case walls until they're finished generating actually makes things feel slower, since the model is changing as you look at it. But I could add that code in if model generation starts taking longer once I start punching out holes for the connectors and generating screw inserts. I also haven't yet tested on slower machines.

Hand Shapes

Not everyone has a camera, and I'd say there's significant spread in the performance of webcams (and user room lighting) possibly causing issues for such a feature. Integrating and processing camera input in browser sounds to me like it'll have performance issues. Additionally, AI NNs don't always create desirable results, and might just require a lot of tweaking of the result (by users). It also seems like a feature that could be built later on top of a good parametric base system. Definitely interesting, and sounds like fun to pursue as additional feature.

Good point about cameras. This makes me think it would be useful to build out a mechanism where you could scan your hand with your phone then continue working off your laptop/desktop. I think the demo I'm using should work on a phone as well.

"Inputs relative to hand."
Sounds a bit like the problem Inverse Kinematics solve: Given a multi-segmented structure it's difficult to specify the individual component orientations to reach a target. Specifying the target location instead, and then solving the multi-segment orientations is more intuitive. In the terms of a keyboard, it might just be easier and more intuitive to position the keys, letting the wrist and fingers solve naturally for the multi-segment orientations. Having to specify multiple relative position/rotation constraints per key sounds exhausting. Also it might not even be possible to solve for such a matrix of constraints.

All good points. I thought about this a bit differently, as I imagined it more as a forward kinematics problem. Key positions could be specified in hand joint angles, then I solve for their positions on the keyboard. And that should be easier mathematically as well.

But you raise a good point in that there's also constraints on key spacing & layout on the keyboard, so this may have a complicated matrix of constraints in the end.

"Generate Diagrams"
Sounds reasonable and desirable. PCBs are probably tricky, given how three dimensional the output could be. A general wiring diagram however could be used to design the PCBs in other software. I can't speak to much on this, since I'm fine with 3D printing the case and hand-wiring it, as it cuts out the delay and cost of getting PCBs made.

Yup at a minimum I'm hoping to design a wiring diagram.

"Ergogen-inspired editor"

Yes yes yes! Starting with a code editor at the beginning was what I was thinking as well. I might just allow for running JavaScript code so that making variables & functions is easier. Having a limited configuration language is why OpenSCAD on its own doesn't work well for the Dactyl, and using a language like JS means no one has to make a clojure generator for the config 😄

Progress

There's still quite a few things left to implement in just simply replicating the manuform generator, and the wall design has some quirks (like I'm not sure if that leftmost tall thumb key has enough clearance from the case).

The wall generation also isn't perfect. Here's the default model. The wall generation is a bit too careful carving out the edges:

But it's progress!

[rianadon]

Progress update: There's lots of new stuff. Rounded edges, auto-generated wrist rest, and new colors.

And of course with a trackball!

[fre-sch]

Impressive progress! Those smooth curves are very appealing! Are those previews based on your own new layout code?

[rianadon]

Yes! They're using the layout code. The walls are being auto-generated from the key configuration.

[rodrigob]

Where does the new code live?

[rianadon]

On my computer for now 😑 There's a few more things I want to implement before I make it public. In the meantime you'll be stuck with me teasing updates.

[rianadon]

There's still a lot I have left to do before I release the next version, so here's an update on the to-do list:

One change hopefully coming soon (to the current site!) is to switch the preview renderer to Manifold.

I've been fixing a few of the idiosyncrasies of the original dactyl generator in V2 (e.g. changing curvatures does not preserve the spacing between keys, the thumb cluster doesn't align to a curved matrix like the other keys do, etc), I've been further diverging from the original design. These differences are very tiny and you may not even notice them, but altogether they've made me start thinking if I want this site to be a replacement of the "V1" site or a new project.

Together with the fact that this update is taking a long time, I want to solve my biggest frustration with V1 which is the excessive holes in the models. I wrote a quick proof of concept replacing OpenJSCAD with Manifold, and if the docs are to believed there should be both an increase in accuracy and speed.

The tasks left for V2 include:

• Add some intelligence to the positioning of screw inserts so that they're not placed directly below the lowest point of the model.
• Speed up the site, especially the first load (some of the libraries are loaded very inefficiently)
• Clean up memory leaks (WebAssembly code does not use garbage collection, so I need to delete objects myself)
• Add support for more component types, including OLED screens and encoders (both the vertical and horizontal types like EVQWGD001)
• Generate trackball mounts for variable radius trackballs
• Smooth the top surface of the keyboard
• Improve the UI (figure out what colors I want to use, give it some more personality, and add a landing page to describe how V2 differs from V1)

Another big task I'd like to complete before the release (or maybe shortly after) is to make the keyboard adapt to your hand size. I have a proof-of-concept going where I can approximately measure the lengths of joints in the finger using a phone/laptop camera. The trouble here is taking those measurements and thinking about how to design a keyboard around them.

My current thought process is to measure the shape of your hand as you let it dangle and wiggle your fingers (hopefully gathering a set of grips that are comfortable) then trying to compute the key staggers and thumb cluster position that put the keys closest to those grasps. Even with a single resting pose of the hand, it's possible to generate a keyboard with the home row sitting directly under each finger. That said, there's still a lot to explore here.

[dohn-joh]

I've been having fun playing with the web configurator and am thankful for your work.

I see you have a lot of new features planned and would like to request a couple features for the upcoming version.

1. Assuming downloading .scad will still be supported, it would be nice if comments were included in the generated .scad file that describe which section of code corresponds to which characteristic. For example // Index Column, // Pinky Stagger, or // Bottom Right Thumb Tenting. Being able to search for and edit different characteristics of the dactyl in openscad would make iterating significantly easier and more efficient.
2. I would like to echo fre-sch's suggestion of having a way to easily export/import/share configuration. This may make the previous request moot because then all configuration could be done in the web configurator by copy/pasting in the data (JSON, YAML, or etc).

[rianadon]

Thank you for the feedback @pujvz! I'm glad you're having fun.

I'm curious after reading your first comment, are you finding it helpful editing values in the .scad files? Are there any configuration options missing in the UI that you were able to tune in the .scad file? A configuration option like the column curvature affects dozens if not hundreds of lines in the file, so I've never considered editing the file myself.

The new version is no longer going to use OpenSCAD but instead the OpenCascade kernel, so there will be no .scad files. I have both a code-based and field-based editor for it. The code one can be copy-pasted in, and the field-based one (like how the current version is) saved the configuration to the URL.

I've also been considering whether I should save the code within the URL as well so it can be easily shared. The code is pretty terse, and once compressed it takes up maybe 400 characters within the URL, which leaves plenty of room for writing your own custom code on top.

[dohn-joh]

Sorry, I was sleep deprived when I wrote that comment, and I am still new and learning more about dactyl generation. I thought that editing the .scad file would be a good idea, but you're absolutely right. It's overly complicated and probably not a good idea. I also discovered today that URL is saveable and shareable.

[hmngwy]

@rianadon this is an interesting evolution of the dactyl generator. a lot of the things you are striving for in this effort is what I'm looking for, from the modernization of the code base to better model generation, not forgetting to mention the curves, are just a few that I align with.

I think one thing you may need help in is IRL build testing. I'm willing to do the test prints, test fitting, etc. I think this would round out an end-to-end test to make this v2 project successful.

[rianadon]

Thank you! Test prints would be a great way to confirm that the models work well.

I'll let you know when it's ready for that—hopefully sometime in the coming weeks.

[rianadon]

I forgot to post an update here, but I made a landing page for V2 of the project (which is called Cosmos): https://ryanis.cool/cosmos/ which has also been linked on the Dactyl configurator website for some time.

The link to access the beta version of Cosmos is behind the Discord linked on the page. This won't be the case when I release the project around August, but for now Discord helps me collect feedback and communicate when I'm going to be adding new features & breaking changes.