Technical Paper | Documentation | X

OpenMind’s OM1 is a modular AI runtime that empowers developers to create and deploy multimodal AI agents across digital environments and physical robots, including Humanoids, Phone Apps, Quadrupeds, educational robots such as TurtleBot 4, and simulators like Gazebo and Isaac Sim. OM1 agents can process diverse inputs like web data, social media, camera feeds, and LIDAR, while enabling physical actions including motion, autonomous navigation, and natural conversations. The goal of OM1 is to make it easy to create highly capable human-focused robots, that are easy to upgrade and (re)configure to accommodate different physical form factors.

Pick Your Runtime

OM1 was originally built in Python, and the Go runtime is a newer, performance-focused implementation. We migrated to Go for lower latency, better performance, efficient concurrency, a smaller memory footprint for edge devices, and simpler deployment as a single Go binary (with the Zenoh C library bundled alongside it).

The Go runtime covers the core agent pipeline, but several capabilities available in the Python runtime are still under active development. For most new development we recommend the Go runtime.

[!NOTE] However, Python version is available at python for use, if preferred. It is now deprecated and will not be maintained.

Capabilities of OM1

• Modular Architecture: Written in Go for performance and seamless integration.
• Data Input: Easily handles new data and sensors.
• Hardware Support via Plugins: Supports new hardware through plugins for API endpoints and specific robot hardware connections to ROS2, Zenoh, and CycloneDDS. (We recommend Zenoh for all new development).
• Pre-configured Endpoints: Supports Text-to-Speech, multiple LLMs from OpenAI, xAI, DeepSeek, Anthropic, Meta, Gemini, NearAI, Ollama (local), and multiple Visual Language Models (VLMs) with pre-configured endpoints for each service.
• Metrics & Observability: Includes a pre-configured Prometheus and Grafana stack to monitor real-time AI pipeline metrics like LLM and ASR latencies.

Architecture Overview

Getting Started

If you are new to OM1, this is the fastest path to a successful first run using the conversation agent.

The conversation agent uses your webcam and microphone to enable natural voice interactions with an AI. The model processes visual and audio inputs and responds through speech.

This quick start uses the default starter configuration to help you understand the OM1 pipeline. It visualizes state updates in the terminal and demonstrates the input-LLM-action flow.

Quick Start (5 Minutes)

1. Install system dependencies.
2. Download the binary or clone the repository.
3. Add your OpenMind API key.
4. Launch OM1 and verify it is running.

1. Install System Dependencies

For macOS:

brew install portaudio ffmpeg

For Linux:

sudo apt-get update
sudo apt-get install -y portaudio19-dev ffmpeg pkg-config

[!TIP] Webcam access is recommended if configuring VLM.

2. Download or Build

Option A: Download Pre-built Binary (Recommended)

No Go installation required. Download the latest release for your platform from the Releases page.

Current pre-built binaries are available for:

• linux-amd64
• linux-arm64
• darwin-arm64
• darwin-amd64

After downloading and extracting the archive, open a terminal in the extracted folder and run:

chmod +x om1
./om1 -config <your-config>

On macOS, if the binary is blocked by Gatekeeper, run:

xattr -d com.apple.quarantine om1 2>/dev/null || true

[!NOTE] The release archive already includes config/, knowledge_base/, and libzenohc files, so you do not need to clone the repository for runtime assets.

Option B: Build from Source

Requires Go 1.25.0+ (installation guide) and make.

git clone https://github.com/OpenMind/OM1.git
cd OM1
make deps
make build

File truncated at 100 lines see the full file

Contributing to OM1

We welcome contributions from the community! OM1 is an open-source project, and we appreciate your help in making it better. Whether you’re fixing bugs, adding features, improving documentation, or suggesting new ideas, your contributions are valuable.

Before contributing, please take a moment to read through the following guidelines. This helps streamline the process and ensures everyone is on the same page.

PRs must clearly state the problem being solved. Changes without a clear problem statement may be closed without review.

Ways to Contribute:

• Report Bugs: If you find a bug, please open an issue on GitHub. Be sure to include:
  • A clear and concise description of the bug.
  • Steps to reproduce the bug.
  • Your operating system and Go version.
  • Relevant error messages or stack traces.
  • Screenshots (if applicable).

• Suggest Features: Have an idea for a new feature or improvement? Open an issue on GitHub and describe your suggestion. Explain the motivation behind the feature and how it would benefit OM1 users. We encourage discussion on feature requests before implementation.

• Improve Documentation: Good documentation is crucial. If you find anything unclear, incomplete, or outdated in the documentation, please submit a pull request with your changes. This includes the README, docstrings, and any other documentation files. Visit OM1 docs, and source code.

• Fix Bugs: Browse the open issues and look for bugs labeled “bug” or “help wanted.” If you want to tackle a bug, comment on the issue to let us know you’re working on it.

• Implement Features: Check the open issues for features labeled “enhancement” or “bounty” or “help wanted”. It’s best to discuss your approach in the issue comments before starting significant development.

• Write Tests: OM1 aims for high test coverage. If you’re adding new code, please include corresponding tests. If you find areas with insufficient test coverage, adding tests is a great contribution.

• Code Review: Reviewing pull requests is a valuable way to contribute. It helps ensure code quality and maintainability.

Out of Scope

• Documentation Translations: Multilingual versions of documentation are not supported. PRs translating docs will be closed.

• Stylistic or Minor Changes: Changes that only affect formatting, variable names, or style without functional improvement are out of scope.

• Trivial or Cosmetic Fixes: Small changes that do not fix bugs or meaningfully improve usability are out of scope.

• Opinion-Driven Refactors: Refactors made solely for personal style or preference without measurable benefit are out of scope.

Contribution Workflow (Pull Requests):

Please open an issue for discussion before submitting PRs that introduce new features, refactors, stylistic changes, or dependency upgrades. PRs of this type without prior discussion may be closed. Once approval is granted, you can start working and open a PR.

1. Fork the Repository: Click the “Fork” button on the top-right of the OM1 repository page to create your own copy.

2. Clone Your Fork with CLI:

    git clone https://github.com/<your-username>/OM1.git
    cd OM1
    

(Replace `<your-username>` with your GitHub username.)

1. Setup Development Environment Refer documentation to setup your development environment.

2. Create a Branch: Create a new branch for your work. Use a descriptive name that reflects the purpose of your changes (e.g., fix-bug-xyz, add-feature-abc, docs-improve-readme).

    git checkout -b your-branch-name
    

1. Make Changes: Make your code changes, add tests, and update documentation as needed.

2. Commit Changes: Commit your changes with clear and concise commit messages. Follow the Conventional Commits specification if possible (e.g., feat: Add new feature, fix: Correct bug in module X, docs: Update README).

    git commit -m "feat: Add support for XYZ"
    

1. Local Testing

   Run the repository checks locally before pushing changes:

    make check
    

Or run individual checks as needed:

    make fmt
    make lint
    make test
    

1. Push Changes: Once all the tests pass locally, push your branch to your forked repository.

    git push origin your-branch-name
    

1. Create a Pull Request (PR): Go to the original OM1 repository on GitHub. You should see a prompt to create a pull request from your newly pushed branch. Click “Compare & pull request.”

2. Write a Clear PR Description: * Describe the purpose of your pull request. * Link to any relevant issues it addresses (e.g., “Closes #123”). * Explain your changes and your design choices. * Include any relevant screenshots or GIFs (if applicable).

3. Request Review: Your pull request will be reviewed by the maintainers. Be prepared to address any feedback or make further changes.

4. Merge: Once your pull request is reviewed and approved, it will be merged into the main branch.

File truncated at 100 lines see the full file