Automatically Generate Sequence Diagrams

Sequence Diagrams are essential for designing and understanding concurrent systems. If you have a system of "actors", whether they are DQMH modules, AF Actors, JKI SMO's, or something else, it is useful to be able to visualize how messages flow over time. Sequence Diagrams are perfect for that. Sequence Diagrams are really useful for designing a system and thinking through the order of operations. Until recently (at least in LabVIEW) you had to manually create them. You couldn't generate them in real time from your actual code while it is running. This kind of left them relegated to a design tool or maybe a documentation tool, as in a documenting your intentions sense. But they are capable of so much more.

Debugging

A big use case for Sequence Diagrams is debugging. You can run into a lot of problems with asynchronous actor systems that can be really hard to diagnose. You end up with race conditions, the wrong message getting sent, things getting sent in the wrong order, deadlocks, circular messaging, etc. Often these are issues where each individual actor behaves correctly in isolation, but the system as a whole seems to be misbehaving. Being able to generate a sequence diagram as your code is running, is really useful. It's especially useful when you can compare it to an as-designed or as-intended Sequence Diagram.

Generating Diagrams

There are different ways of creating these diagrams. Fabiola put me onto websequence diagrams.  Stefan and some others are big fans of plantuml. Both of these tools allow you to type in some text and they use that text to generate a sequence diagram. Out of the box these are manual tools. However because they take text input, they can be scripted. We can write some script that automatically generates that text.

A First Attempt

The first attempt at generating sequence diagrams from running code was a few years ago. Someone had figured out a solution for Actor Framework. It required you to turn on the DETT traces for AF. Then you could capture the log from the DETT and run it through a Python script that would generate text that you could then paste into plantuml. It worked, but it wasn't super-generic, and it required jumping through a few hoops.

Lumos

Enter Lumos. That AF tool inspired Olivier and he went and created a tool for doing something similar with DQMH. He did a demo at a conference a year or two ago. THe demo went over really well. There was a lot of demand, but it wasn't quite a finished project. It has now been taken on by the DQMH Consortium.

Tooling is Key

The DQMH Consortium is trying to raise some funds to polish it up and release it as a paid toolkit to raise money for further DQMH development. As you might expect from them, they have invested heavily in a bunch of scripting tools to make the tool easier to use and more valuable. They have scripting tools to automatically add all the hooks. Lumos has a GUI tool that generates an ASCII art version of the diagram in real time and allows you to filter events based on types of events or participants. This can be very useful in larger systems where you have a lot of things going on and want to focus in on certain interactions.

Documentation  Generation and Testing

Lumos also has hooks for calling it programmatically. So as part of your CI/CD pipeline, you could use something like SLL Drona to automate running your program and then capture the output and use it in your documentation. You can also combine that output with an approval testing framework to write automated tests around the sequence diagrams. The ability to generate sequence diagrams programmatically opens up a lot of doors.

Other Frameworks

After talking to the Consortium, the first version is specifically geared towards DQMH, but with some minor hoop jumping it will work with any framework. Basically, all the tools for integrating Lumos with other frameworks are there, they just aren't automatically exposed and they don't have polished tooling around them. Version 2 will be much more generic and make it very easy to integrate with any framework by exposing those methods and maybe adding some more generic tooling.

More Info

The tooling around Lumos and its ability to work with other frameworks make it much more useful than the AF/DETT/Python implementation that inspired it. The DQMH Consortium released a video describing some of the features. Be sure to check it out.

How Can I Support This?

The DQMH Consortium has started a Kickstarter. You can donate there. As usual with KickStarter if you donate enough you'll get a license to the final product and a bunch of other perks. If you use AF, DQMH, or SMO's you should consider supporting it. It is going to be a very useful tool. There is a bunch of information including a video on the Kickstarter site. You should definitely check it out.