Error message
Any components that appear in this error message aren’t necessarily the cause of the error. They may only be named because they were running when the error occurred.
Explanation
The Maia runner instance running your pipeline stopped reporting to Maia before the work finished, so Maia marked it offline and failed the component. The error is a symptom of the instance going away.Potential causes
- The container or virtual machine restarted, crashed, or was replaced while the component was running.
- The instance was stopped by the platform it runs on, often because it ran out of memory. A container killed this way can disappear without the Maia runner reporting an out-of-memory error.
- Heavy Python scripts or many parallel tasks used more memory or CPU than the instance has.
- A network interruption between the Maia runner and Maia lasted longer than the heartbeat window.
Resolve the error
- Check whether the Maia runner restarted or was replaced around the time of the failure.
-
In your container platform, such as Kubernetes, Amazon Elastic Container Service (ECS), or Azure Container Apps, look at the instance’s events and logs. Exit code
137or anOOMKilledreason means the instance ran out of memory. - If the failed component is Python Script, move the work into Snowflake with Python Pushdown, or give the Maia runner more memory.
- Lower the number of pipelines and loop iterations running in parallel on that Maia runner.
- Re-run the pipeline. If your pipelines are safe to repeat, set up a retry for scheduled runs.
-
If you still need help, contact Maia Support and include the items listed below.
- Your Maia runner‘s platform (Kubernetes, ECS, or Azure Container Apps) and its memory limit.
- Container exit codes or restart events from the time of failure.
Prevent future errors
To prevent this error occurring again:- Run at least two Maia runner instances so work can continue if one is lost.
- Set memory limits with headroom above your largest job.
- Design pipelines so a repeat run doesn’t duplicate data.