> ## Documentation Index
> Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming runner logs



\<table><tbody><tr><td>Runner type</td>
            \<td>Streaming</td></tr>
          \<tr><td>Runner platform</td>
            \<td /></tr></tbody></table>

The activity of your Streaming runner can be logged in your cloud provider's logging service, either [AWS CloudWatch](#aws-cloudwatch) or [Azure Log Analytics](#azure-log-analytics), to help you address any issues you encounter. Analyzing the logs generated by your Streaming runner enables you to identify patterns, troubleshoot issues, and gain insights into the behavior of your deployment.***## AWS CloudWatchYou can retrieve, inspect, and analyze log data using AWS CloudWatch in the AWS console.By default, logs are kept for an indefinite time, but you can set up a retention period based on your requirements.### Prerequisites- You must have a valid AWS account to access and use AWS CloudWatch.
- Your [Streaming runner](https://docs.maia.ai/docs/streaming/create-streaming-runner.md) must be deployed in the instance for which you want to generate logs. The Streaming runner captures and forwards log data to AWS CloudWatch for monitoring and analysis.
- Your AWS account must have the necessary IAM (Identity and Access Management) permissions to access CloudWatch and retrieve logs. At a minimum, the account should have the `cloudwatch:DescribeLogGroups` and `cloudwatch:FilterLogEvents` permissions for the specific log group and log streams you want to access.
- Configure a log group in CloudWatch to store the logs generated by the Streaming runner.
- Configure the log retention settings for the log group to ensure that logs are retained for the desired period.### How to generate logs in AWSTo generate logs from AWS CloudWatch for a deployed Streaming runner in an instance, follow these steps.1) In the [AWS Management Console](https://aws.amazon.com/console/), navigate to the **CloudWatch** service from the services menu or from the search bar at the top of the console screen.

2) In the **CloudWatch** dashboard, locate and click **Log Groups** in the left-hand navigation menu under the **Logs** category.

3) In the **Log Groups** page, search for the log group associated with your Streaming runner's logs. The log group should have a name or pattern specific to your deployment, which you will have set when you installed the Streaming runner.

   > [!NOTE]
   >
   > If a log group for the Streaming runner does not already exist, create a new log group by following the process in [Log groups](https://docs.maia.ai/docs/streaming/aws-streaming-runner-manual-setup.md#log-groups-optional).

4) Click on the log group to access its details. You will see a list of log streams within that group.

5) Identify the log stream that corresponds to the specific instance where your Streaming runner is deployed. Click on the log stream to view its logs.

6) Within the log stream, you can locate specific logs by applying filters, searching for keywords, or adjusting the time range.***## Azure Log AnalyticsIn the Azure console, you can retrieve, inspect, and analyze log data using the Azure Log Analytics service. Azure Log Analytics provides a centralized location for log storage, analysis, and visualization, enabling you to monitor and troubleshoot your Azure environment effectively.Azure Log Analytics also allows you to create custom dashboards and set up alerts based on log data. These features can help you monitor and receive notifications for specific events or log conditions.### Prerequisites- You must have a valid Azure subscription to access and use Azure Log Analytics.
- You must have Azure Container Instances (ACI) or an Azure Kubernetes Service (AKS) cluster where the Streaming runner is deployed in a container. The container must be running and generating logs.
- Ensure that an Azure Log Analytics workspace has been created. This should have been created when you [installed the Streaming runner](https://docs.maia.ai/docs/streaming/azure-streaming-runner-install.md#resources-deployed).
- You must [enable container monitoring](https://learn.microsoft.com/en-us/azure/azure-monitor/vm/vminsights-enable-portal) for the Azure Container Instances or Azure Kubernetes Service cluster where the Streaming runner is deployed. This ensures that the logs generated by the container are collected and forwarded to Azure Log Analytics.### How to generate logs in AzureTo generate logs from Azure Log Analytics for a deployed Streaming runner in an instance, follow these steps.1) In the [Azure Portal](https://portal.azure.com), navigate to the **Log Analytics** service from the services menu or from the search bar at the top of the portal screen.
2) In the **Log Analytics** dashboard, search for the workspace associated with your Streaming runner's logs. The workspace should have a name or pattern specific to your deployment. Click the workspace name to enter that workspace.
3) In the Log Analytics workspace, click **Activity Log**. By default, you will see the **Logs** section where you can run queries and view log data.
4) In the **Logs** section, you can write and [execute queries](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/get-started-queries) using the **Azure Monitor Query Language (KQL)** to retrieve specific logs. You can filter logs based on time range, specific instance, or other relevant parameters. See the example below.### Query example```kql theme={null}
// Retrieve logs with specific table
  ContainerInstanceLog_CL
  | project TimeGenerated, Message
```Where `TimeGenerated` and `Message` are fields from the `ContainerInstanceLog_CL` table.This query will retrieve the `timestamp` (TimeGenerated) and the `log message` (Message) from the logs generated by container instances.***## Export logsWith the export logs feature enabled, logs from the Streaming runner are automatically sent to Matillion's systems for support. They are stored in the same cloud provider and geolocation as your Maia account.### Enable log exportTo enable log export, set the `EXPORT_LOGS` optional Streaming runner environment variable to `true`. By default, `EXPORT_LOGS` is set to `false`.### Log retentionLogs are stored for 30 days, after which they are removed.### Stored dataThe same information that is visible in your local Streaming runner logs will be available to Matillion for support purposes. Matillion cannot access any data originating from your data sources or cloud data warehouse.### Further informationFor more information on data gathering at Matillion, read [Data gathering](https://docs.maia.ai/docs/security/data-gathering.md#types-of-data) and our [Security overview](https://docs.maia.ai/docs/security/security-overview.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.