Idem is a powerful tool that allows for efficient cloud resource management. In this post, we will explore the three main components of Idem: States, Exec, and Tool. Understanding these components is essential for effectively using Idem.
States - Managing Cloud Resources
State modules in Idem are typically used to manage cloud resources. They are stored under my_project/states. A state, such as aws.ec2.instance.present, can ensure that an AWS EC2 instance exists with the parameters defined in your SLS file. Managing cloud infrastructure is as simple as tweaking an SLS file and then running it with `idem state my_states.sls`.
By convention, state functions contain a "present" and "absent" funciton. "present" ensures that a resource exists and has the attributes consistent with the present parameters. "absent" ensures that a resource does not exist. These functions, as well as "describe" *must* exist in states that implement the resource contract.
State functions should be idempotent, meaning you can run them multiple times and they resolve to a stable state. States modules take at least hub, ctx, and name as parameters, with additional keyword parameters as needed.
Exec - Functionality Callable from the CLI
Exec modules are stored in my_project/exec and are callable from the CLI using the command `idem exec <exec_ref>`. Exec modules usually make direct calls to APIs and do the heavy lifting in Idem. By convention, manually written exec modules should contain Create, List, Get, Update, and Delete functions (sometimes called CRUD operations).
Some exec modules are raw credentialed wrappers of an API. For example, in idem-aws everything under hub.exec.boto3 makes a direct call to the boto3 libraries, adding its namespaces to the hub under a ReverseSub, while everything under hub.exec.aws contains manually written CRUD operations.
Tool - Intermediary Operations and Utilities
Tool modules are stored under my_project/tool and are used for intermediary operations that are not useful on the CLI. These functions require only the hub parameter, with additional parameters as needed.
For example, in idem-aws, Tool functions are used to normalize tags into a plain dictionary for consistent usage across the platform.
Common Pitfalls and Best Practices
Some common pitfalls include neglecting to write complete modules with present, absent, describe, etc., and placing code in the top level of my_project/tool, my_project/exec, or my_project/states instead of using nested namespaces for organization.
Remember that States, Exec, and Tool plugins get merged together on the hub, so it is essential to keep your code organized in redundant namespaces to avoid collisions.
Testing Strategies and Tools
For testing Idem plugins, pytest-idem is commonly used. It offers tools for running the Idem CLI reliably with credentials and extending the Python environment as needed with tests/tpath. Code that is use exclusively for testing does not belong in the production plugin space. tests/tpath contains states/exec/tool plugins that are loaded by pytest-idem exclusively for testing.
Performance Optimization and Scalability
Ensuring scalability and performance is crucial. To achieve this, present, absent, and describe functions in States, as well as get, list, create, update, and delete operations in Exec modules, should be asynchronous (async). This allows Idem to run many states in parallel, optimizing the time it takes to make requests to cloud APIs.
Community Contributions and Custom Functionality
One of the advantages of Idem being built on POP is that you don’t need permission from Idem maintainers to add functionality. Create your project and add states under your own namespaces, like your_project/states/aws/my_aws_state.py. This allows you to contribute and add your own functionality without needing anyone’s approval.
Conclusion
Understanding the main components of Idem - States, Exec, and Tool - and how they interact is key to harnessing its full potential. Following best practices, keeping code organized, optimizing for performance, and considering security are all critical elements of using Idem effectively.