FlexSim 2022: Reinforcement Learning + Experimenter Improvements
FlexSim 2022 introduces new features for artificial intelligence and machine learning applications, a significant update to the Experimenter, and expanded chart pinning options.
The Reinforcement Learning tool combines the interface, files, and documentation you will need to use a FlexSim model as an environment for training and evaluating reinforcement learning algorithms.
At its core, this tool provides the features needed for a reinforcement learning algorithm to communicate with FlexSim. You’ll find a simple yet powerful interface for both training the algorithm and using the AI-trained model in a real production environment. Easily set up your observation and action spaces, add a reward function, and specify decision events. The Reinforcement Learning tool is even flexible enough to be able to interface with any machine learning environment you’d like.
When you’re ready to get started with reinforcement learning, there is a complete tutorial in the FlexSim documentation. You will set up a pre-built algorithm that will learn to minimize changeover times as it figures out which item to pull next. Once you’re finished with the example, you’ll have a trained reinforcement learning model—and you’ll be able to evaluate the AI’s decisions in your simulation environment and even use the trained AI to make predictions.
Experimenter Results Database File
All experiment results data is now saved in a Sqlite database file, which by default has the same name as the model file and is saved in the same directory. This offers two advantages over previous versions of FlexSim:
- This file does not use RAM to store your results, so you can run experiments with many scenarios and many replications without eating up system resources.
- All experiment data is stored in this separate database file on your hard drive, which means the model file itself will be smaller.
The former “Scenarios” tab has been redesigned and renamed “Jobs” (this is also the new term when discussing simulation experiments in FlexSim). Simply click the green plus sign to add a new Job to the model. Each Job represents the parameters and scenarios that make up the experiment, whether it’s a normal simulation experiment or an optimization experiment using the OptQuest add-on. The separate Optimizer tabs are no longer necessary and have been removed.
This update also means you can have multiple experiments (Jobs) in a single model.
Control Over Replications
As you change a Job, the “Run” tab will update to show the scenarios and replications that make up the Job. There is also a unique identifier for every scenario and replication. These unique identifiers bring several convenient new features for your experiments:
- You no longer need to re-run every replication each time you change the experiment. If you add more scenarios and replications, they will be added as “proposed” replications alongside the already completed ones—to be completed with the next batch.
- You can delete the data from a single replication and re-run just that replication.
Finally, there is a Test Tube button found when hovering over each unique replication. Click this button and you can apply the parameters from that specific scenario and replication to the model.
Other Experimenter Features
- Batch Experimentation: FlexSim now supports running multiple Jobs with a single click. You can select as many Jobs as you’d like and have them run sequentially as a batch.
- Easily Extract State Files: If you choose to save a state file after each replication (an option now found in the “Advanced” tab), these states will be saved in the results database file. You can easily access and extract these files from a new tab in the Performance Measure Results.
Expanded Chart Pins
The Pin button , which is found in the Statistics panel in an object’s properties, now has additional options for chart type when you pin to a Dashboard. This works with all pins in FlexSim, including those on non-3D objects such as a Process Flow activity. Tracked variables and labels can now also be pinned to Dashboards.
- There are several new charts, including Label charts and Tracked Variable charts.
- MTBF/MTTR time fields have been updated to show unit conversion options.
For more detailed release notes: https://docs.flexsim.com/en/22.0/Reference/ReleaseNotesAndHistory/