> Documentation index: https://docs.mayur.live/llms.txt # Processing Feedback and Scheduled Cleanup Understand and monitor Inngest background processing for transcription, sentiment analysis, and scheduled housekeeping. Monitor and operate the Voiceflow background jobs that process uploaded audio recordings and run routine maintenance. Background jobs are handled through Inngest, mounted at `/api/inngest`. ## Pipeline Execution Steps Feedback processing begins when the `feedback/uploaded` event fires after an upload is completed. The `processFeedback` function manages processing through three discrete steps: ```text claim ─> speech (Sarvam x2) ─> sentiment (LLM) ─> PostgreSQL ``` 1. **`claim`**: The function takes ownership of the feedback record by updating its status to `PROCESSING` and clearing any existing `errorCode`. If the record is already moved beyond `UPLOADED` or `PROCESSING`, the step skips further execution. 2. **`speech`**: The function checks whether a transcription record already exists. If missing, it downloads the audio payload from storage using `getObjectBytes`, invokes `transcribe`, and stores the resulting transcript in the database. 3. **`sentiment`**: The function passes the English transcript to `analyseSentiment`. It updates the database in a transaction that marks the feedback row as `COMPLETED` with a `completedAt` timestamp. ### Retries, Throttling, and Failures The `processFeedback` job uses the following execution constraints: * **Concurrency**: Restricted to a limit of `1` keyed by `event.data.feedbackId`. Duplicate or rescued events wait until earlier runs finish, then stop if the record has already moved on. * **Throttling**: Capped at `25` calls per `1m` to respect upstream Sarvam limits. * **Retries**: Up to 3 retries for transient errors. * **Failure Handling**: If the job runs out of retries or encounters non-retriable pipeline errors (`SPEECH_REJECTED`, `NO_SPEECH`, or `ANALYSIS_INVALID`), the feedback row status is updated to `FAILED` and sets the `errorCode`. Other unhandled failures record the error code `PROCESSING_FAILED`. ## Speech Transcription with Sarvam The `transcribe` service interacts with Sarvam's Speech-to-Text API (`https://api.sarvam.ai/speech-to-text`) using the `saaras:v4` model. For each recording, `transcribe` performs two parallel calls using the provided audio buffer and MIME type: 1. **Original transcript**: Runs Sarvam in `transcribe` mode to transcribe the speech in the indicated language (or `"unknown"` if no `languageHint` is set). 2. **English transcript**: Runs Sarvam in `translate` mode to generate an English translation. Each call enforces an abort timeout of 20 seconds. ### Error Handling in Transcription * **Empty Transcripts**: If either the original or English transcript is empty after trimming whitespace, the service throws a non-retriable `NO_SPEECH` error. * **Client Errors (HTTP 400, 403, 422)**: Bad requests, invalid authentication keys, or unprocessable audio throw a non-retriable `SPEECH_REJECTED` error. * **Rate Limits (HTTP 429)**: Throws a `RetryAfterError` requesting a 30-second backoff (`SPEECH_RATE_LIMITED`). * **Network Errors and Timeouts**: Throws `SPEECH_UNAVAILABLE` to trigger standard Inngest retries. Because transcripts are stored immediately during the `speech` step, downstream failures during sentiment analysis do not re-run or re-bill Sarvam calls on retry. ## Sentiment Analysis The `analyseSentiment` service processes the generated English text using the OpenRouter AI SDK provider and the model specified in `SENTIMENT_MODEL` (defaulting to `openai/gpt-4o-mini`). It runs with prompt version `v3`. To protect against prompt injection, the transcript is placed inside `` tags and declared strictly as input data within the system instructions. ### Structured Output and Schema The LLM returns an object validated against a Zod schema containing: * **`label`**: Enum of `"POSITIVE"`, `"NEUTRAL"`, `"NEGATIVE"`, or `"MIXED"`. * **`score`**: A polarity value between `-1` and `1`. An internal helper (`alignScore`) ensures the sign matches the label (positive values for `"POSITIVE"`, negative values for `"NEGATIVE"`). * **`summary`**: A single sentence summarizing the feedback, capped at 25 words. * **`topics`**: An array of identified topics constrained to: `"food quality"`, `"delivery time"`, `"packaging"`, `"price"`, `"staff behaviour"`, `"hygiene"`, `"order accuracy"`, `"missing items"`, `"portion size"`, or `"other"`. * **`urgent`**: A boolean flag set to `true` for complaints related to health, safety, or serious issues. If the model response fails schema generation (`NoObjectGeneratedError`), the step logs the error and raises a non-retriable `ANALYSIS_INVALID` error. ## Scheduled Housekeeping Cleanup The `housekeeping` function runs on a scheduled cron trigger every 10 minutes (`*/10 * * * *`) with up to 2 retries. It runs three tasks sequentially as independent steps so that a failure in one step does not repeat the others: ```text expire-abandoned-uploads ─> requeue-stuck-uploads ─> delete-unneeded-audio ``` ### Cleanup Tasks | Step / Service Function | Execution Threshold | Batch Size | Action Taken | | --- | --- | --- | --- | | `expireAbandonedUploads` | `status = "UPLOADING"` older than 15 minutes (`UPLOAD_TIMEOUT_MS`) | All matching rows | Marks rows with status `EXPIRED` and error code `UPLOAD_EXPIRED`. Uses a single conditional query to prevent races with customer confirmations. | | `requeueStuckUploads` | `status = "UPLOADED"` unchanged for more than 5 minutes (`STUCK_AFTER_MS`) | 100 rows (`BATCH_SIZE`) | Re-emits `feedback/uploaded` events to Inngest using an event ID formatted as `{id}-{attempt}-rescue-{timestamp}` to restart stalled processing. | | `deleteUnneededAudio` | Audio not yet deleted (`audioDeletedAt` is null) and status is `EXPIRED` or record is older than 30 days (`AUDIO_RETENTION_DAYS`) | 100 rows (`BATCH_SIZE`) | Deletes audio files from storage via `deleteObjects`, then sets `audioDeletedAt` to the current timestamp. Transcripts and sentiment rows remain in the database. | :::note If `deleteUnneededAudio` fails during file deletion, the database rows remain unflagged (`audioDeletedAt: null`), causing the operation to retry automatically on the next scheduled run. :::