# Bulk Data Platform APIs

## Interacting with Bulk Data Platform (BDP) APIs

This sequence diagram outlines the interactions between the User, Motorweb (MW), BDP, and MW Storage during the batch processing workflow:

1. **Authenticate**:
   * User sends a POST request to MW for authentication, receiving an access token in response.
2. **Get Batch Service ID**:
   * User requests the available batch services from BDP, receiving service details (UUID, name, description).
3. **Submit Batch Request**:
   * User submits a batch request to BDP, receiving a signed URL, expiry, and batch ID for file upload.
4. **Optional - Fetch Batch Template**:
   * User can optionally fetch a CSV template for the selected batch service.
5. **Upload Input File**:
   * User uploads the input file to MW Storage using the signed URL from BDP.
6. **Optional - Poll Status**:
   * User can periodically check the batch status every 15 minutes by polling BDP for updates.
7. **Optional - Delete Pending Batch**:
   * If necessary, the user can delete a batch in the pending state by sending a DELETE request to BDP.
8. **Get Result File URL**:
   * Once the batch is complete, the user requests the batch status from BDP, retrieving the result file URL.
9. **Download Result File**:
   * User downloads the result file from MW Storage using the provided URL.
10. **Get Failed Transactions**:
    * User can request a CSV file containing details of any failed transactions in the batch from BDP.

{% @mermaid/diagram content="sequenceDiagram
autonumber
participant user as User
participant mw as Motorweb
participant bdp as BDP
participant storage as MW Storage

```
%% Authenticate user
note over user,mw: Authenticate
user->>mw: POST auth/v1/token
mw-->>user: 200: <...,access_token>

%% Get Batch Service ID
user->>bdp: GET /v1/batch-services
note over user,bdp: Get Batch Service ID
bdp-->>user: 200: <UUID, name, description>


%% Submit a Batch Request
user->>bdp: POST /v1/batches
note over user,bdp: Submit Batch Request
bdp-->>user: 200: <Signed URL, expiry, batchId>

%% Optionally fetch the Inpu template of the batch
opt Optional fetch template for Batch Service as CSV
    user->>bdp: GET /v1/batch-services/{batchServiceID}/template
    note over user, bdp: Batch Service template as CSV
    bdp-->>user: 200 <csv>
end

%% Upload Input file
user->>storage: Upload Input File
note over user, storage: Using Signed URL from previous response, upload to bucket

%% Optionally fetch the status of the batch
opt Poll Status
    loop Check status for completion every 15mins 
      user->>bdp: GET /v1/batches/{batchID}/reports
      note over user, bdp: Batch Status
      bdp-->>user: <status, url,...>
    end
end

%% Optionally Delete a Pending batch
opt Optional Delete a Batch in 'Pending' Status
    user->>bdp: DELETE /v1/batches/{batchID}
    note over user, bdp: Delete a pending batch
    bdp-->>user: 200
end

%% Get Result File URL
user->>bdp: GET /v1/batches/{batchID}/reports
note over user, bdp: Batch Status
bdp-->>user: <status, url,...>

%% Get Result File URL
user->>storage: Download via URL provided in reports endpoint
note over storage, user: Fetch results of after Batch completion

%% Get Failed Transactions Files
user->>bdp: GET /v1/batches/{batchId}/errors/download
note over bdp, user: Batch errors
bdp->>user: <CSV file containing Batch errors>" %}
```