Here’s a clear and simple guide to resolving the issues you might encounter with ingestion jobs and vector parsing failures in AWS Bedrock.
If your ingestion job gets stuck in the STOPPING state and doesn’t move to STOPPED, there isn’t a way to force it to stop directly through the available API. This situation has happened to others, and a common workaround is to temporarily remove the permissions associated with the IAM role used by the Knowledge Base. By doing so, you could cause the job to fail and ultimately reset its state. If other services depend on this role, you might want to remove only the Bedrock-specific permissions and then restore them afterward.
If removing permissions doesn’t resolve the issue, your best option is to contact AWS Support. Keep in mind you’ll need a paid support plan for this. Support engineers have more tools and visibility into the inner workings of your jobs and can help fix the problem directly.
For those experiencing issues with parse failures during vector ingestion in OpenSearch 3.5, the problem might be with the way documents are processed before they are indexed, rather than how the index itself is configured. Your index settings, including the knn_vector dimension and type, seem correct. The error previews show valid vectors, so the problem likely occurs earlier in the process.
Here are some steps to troubleshoot this:
- Update to OpenSearch 3.6.0 or newer: The latest release notes mention a fix related to vectors being incorrectly returned during bulk indexing. If the vector parsing path triggers this bug, upgrading might resolve the issue.
- Test with an older OpenSearch version: Temporarily switching to an earlier version, like 2.x, with the same knowledge base and parsing setup can help determine if the problem is specific to version 3.5.
- Examine CloudWatch logs: Enable logging for your ingestion jobs. This can reveal more detailed error messages that are not visible through the console, helping you pinpoint where the failure occurs.
- Use an alternative vector store temporarily: If you need to continue ingestion immediately and can’t update right now, consider creating a new data source that points to another supported storage, like OpenSearch Serverless. This allows you to keep working while investigating the root cause.
Following these steps should help you troubleshoot and resolve the problems efficiently.