Skip to main content

MinIO Connector

Description​

The MinIO Connector is designed to interact with MinIO object storage. It allows users to authenticate, connect to their bucket, and retrieve file content. The connector supports various file formats and can fetch files either from the root directory or a specified folder. Users can interact with the extracted file content using ACE Chat and Search.

Table of Contents​

Input Parameters​

Required Parameters​

  • Access Key

    • Description: The access key for MinIO
    • Example: minio-access-key

  • Secret Key

    • Description: The secret key for MinIO
    • Example: minio-secret-key

  • Bucket Name

    • Description: The name of the bucket to fetch files from
    • Example: my-bucket

  • Endpoint URL

    • Description: The API URL for MinIO (include port)
    • Example: http://127.0.0.1:9000

Optional Parameters​

  • Folder Path

    • Description: The specific folder to fetch files from. Leave blank or use / to fetch files only from the root folder. Nested files are skipped unless explicitly in the folder.
    • Example: documents/reports

  • Region

    • Enter region if necessary
    • Example: us-east-1

Example Input​

Fetching Files from Root Folder​

To fetch files from the root folder:

  • Access Key: minio-access-key
  • Secret Key: minio-secret-key
  • Bucket Name: my-bucket
  • Endpoint URL: http://127.0.0.1:9000
  • Folder Path: /

Fetching Files from a Specific Folder​

To fetch files from a specific folder:

  • Access Key: minio-access-key
  • Secret Key: minio-secret-key
  • Bucket Name: my-bucket
  • Endpoint URL: http://127.0.0.1:9000
  • Folder Path: project-data

Notes​

  • Ensure the endpoint_url points to the API port of your MinIO deployment (e.g., 9000) and not the console port (e.g., 9001).

  • If invalid credentials are provided, an appropriate error message will be displayed:

    Error: Authentication failed. Please check your access key and secret key.

  • Nested files are skipped unless explicitly part of the folder path provided.

  • If the folder path does not exist or is empty, an appropriate error will be raised:

    Error: The folder 'non-existent-folder' does not exist or is empty.

Supported File Types​

The connector supports the following file formats:

  • Text Files (.txt)
    • Extracts plain text content.
  • Word Documents (.docx)
    • Extracts textual content from Word files.
  • PowerPoint Presentations (.pptx)
    • Extracts slide content.
  • Excel Spreadsheets (.xlsx, .xlsm)
    • Extracts data from sheet cells.
  • PDF Documents (.pdf)
    • Extracts textual content.

Setting Up MinIO (Quick Reference)​

If you're new to MinIO, here's a quick guide to set it up:

  1. Install MinIO Server

    • Download from MinIO's official website
    • Or use Docker:
      docker run -p 9000:9000 -p 9001:9001 --name minio \
        -v /mnt/data:/data \
        -e "MINIO_ROOT_USER=minio-access-key" \
        -e "MINIO_ROOT_PASSWORD=minio-secret-key" \
        quay.io/minio/minio server /data --console-address ":9001"

  2. Access MinIO Console

    • Navigate to http://127.0.0.1:9001 in your browser
    • Login with your access key and secret key

  3. Create a Bucket

    • Click on "Create Bucket" in the console
    • Enter your desired bucket name
    • Configure access permissions as needed

  4. Upload Files

    • Navigate to your bucket
    • Use the upload button to add files
    • Create folders as needed for organization

Troubleshooting​

IssueSolution
Connection refusedCheck if MinIO server is running and the endpoint URL is correct
Authentication failedVerify access key and secret key
Bucket not foundEnsure the bucket name is spelled correctly and exists
No files foundCheck if the folder path is correct or if the bucket is empty
File type not supportedVerify that your files are in one of the supported formats

Best Practices​

  • Organization: Use a clear folder structure in your MinIO buckets to make data retrieval more efficient.
  • Permissions: Set appropriate bucket policies to ensure data security.
  • File Types: Store files in supported formats to ensure proper content extraction.
  • Monitoring: Regularly check the connector logs for any issues or errors.
  • Updates: Keep your MinIO server updated to ensure compatibility with the connector.

FAQ​

Q: Can I connect to multiple MinIO buckets?
A: Yes, you can create separate connector configurations for each bucket.

Q: Is there a limit to the file size that can be processed?
A: While there's no hard limit, very large files may take longer to process. We recommend files under 100MB for optimal performance.

Q: Can I use MinIO with SSL/TLS?
A: Yes, use an HTTPS endpoint URL and provide the appropriate certificates if required.

Q: What happens if a file can't be processed?
A: The connector will log an error but continue processing other files.

Q: Does the connector maintain file versions?
A: No, the connector retrieves only the latest version of each file.