> ## Documentation Index
> Fetch the complete documentation index at: https://docs.condense.io/llms.txt
> Use this file to discover all available pages before exploring further.

# HTTP (Input, Stream)

# Overview

The HTTP Input Connector enables Condense to ingest data from external HTTP or HTTPS endpoints and publish it to Apache Kafka topics for real-time stream processing.

The connector supports both **Polling** and **Webhook** modes, allowing applications to either periodically fetch data from REST APIs or receive events instantly through HTTP requests. It also supports multiple authentication methods, SSL/TLS security, configurable polling intervals, and flexible data formats, making it suitable for integrating external systems with streaming pipelines.

***

# Key Features

The HTTP Input Connector provides the following capabilities:

* Supports Polling and Webhook modes
* Ingests data from HTTP and HTTPS endpoints
* Publishes incoming events to Kafka topics
* Supports multiple authentication mechanisms
* Supports One-Way and Two-Way SSL (Mutual TLS)
* Supports JSON, String, Avro, and Binary payloads
* Configurable polling intervals
* Configurable batch processing
* Retry and error-handling strategies
* No-code configuration through the Condense UI

***

# Supported Input Modes

| Mode        | Description                                                                          |
| ----------- | ------------------------------------------------------------------------------------ |
| **Polling** | Periodically calls an HTTP endpoint and publishes the response to Kafka.             |
| **Webhook** | Receives HTTP POST requests and immediately streams the received payload into Kafka. |

***

# Supported Authentication

The connector supports multiple authentication methods depending on the target HTTP service.

| Authentication        | Supported |
| --------------------- | :-------: |
| None                  |     ✅     |
| Basic Authentication  |     ✅     |
| JWT Bearer Token      |     ✅     |
| API Key               |     ✅     |
| OAuth 2.0             |     ✅     |
| Digest Authentication |     ✅     |

***

# SSL/TLS Support

The connector supports secure communication with HTTPS endpoints.

| SSL Mode                 | Description                                   |
| ------------------------ | --------------------------------------------- |
| HTTP                     | Unencrypted communication                     |
| One-Way SSL              | Validates the server certificate              |
| Two-Way SSL (Mutual TLS) | Validates both client and server certificates |

***

# Supported Payload Formats

The connector supports the following data formats:

* JSON
* String
* Avro
* Binary

***

# Typical Use Cases

The HTTP Input Connector is commonly used for:

* Polling REST APIs
* Receiving webhook notifications
* Collecting IoT events
* Integrating third-party SaaS applications
* Importing business events into Kafka
* Event-driven microservices
* Real-time data ingestion

***

# Prerequisites

Before configuring the connector, ensure:

* The HTTP or HTTPS endpoint is accessible.
* Required authentication credentials are available.
* SSL certificates are available if HTTPS is used.
* A Kafka topic exists to receive incoming events.
* Appropriate permissions are available to deploy connectors.

***

# How to Configure the HTTP Input Connector

## Step 1. Navigate to Connectors

1. Log in to Condense.
2. Navigate to **Connectors**.
3. Click **Create Connector**.
4. Select **HTTP Input Connector**.

> **Add Screenshot:** Connector Selection

***

## Step 2. Configure Basic Information

Provide the following information:

* Connector Name
* Description (optional)
* Target Workspace

***

## Step 3. Configure the HTTP Endpoint

Enter:

* Endpoint URL
* HTTP Method (if applicable)
* Request Headers
* Query Parameters (optional)

> **Add Screenshot:** Endpoint Configuration

***

## Step 4. Select Input Mode

Choose one of the following:

### Polling Mode

Configure:

* Polling Interval
* Request Timeout
* Batch Size

### Webhook Mode

Configure:

* Webhook Endpoint
* Listener Configuration

> **Add Screenshot:** Mode Configuration

***

## Step 5. Configure Authentication

Select the required authentication type.

Supported options include:

* None
* Basic Authentication
* JWT
* API Key
* OAuth2
* Digest Authentication

Provide the required credentials based on the selected authentication mechanism.

> **Add Screenshot:** Authentication Settings

***

## Step 6. Configure SSL

If HTTPS is used, configure:

* One-Way SSL
* Mutual TLS (Two-Way SSL)

Upload certificates if required.

> **Add Screenshot:** SSL Configuration

***

## Step 7. Configure Kafka Settings

Specify:

* Output Kafka Topic
* Message Format
* Key Configuration (optional)

***

## Step 8. Configure Error Handling

Select one of the available strategies:

| Strategy | Description                           |
| -------- | ------------------------------------- |
| Retry    | Retries failed requests automatically |
| Skip     | Ignores failed records                |
| Fail     | Stops connector execution             |

Configure retry interval and maximum retries if Retry is selected.

***

## Step 9. Deploy the Connector

1. Review all configurations.
2. Click **Deploy Connector**.
3. Monitor deployment from the **Pipelines** page.

***

# Editing the HTTP Input Connector

1. Navigate to **Pipelines**.
2. Select the deployed HTTP Input Connector.
3. Click **Edit Configurations**.
4. Update the required settings.
5. Click **Deploy Connector** to apply the changes.

> **Add Screenshot:** Edit Configuration

***

# Deleting the HTTP Input Connector

1. Navigate to **Pipelines**.
2. Select the deployed connector.
3. Click **Delete Connector**.
4. Confirm the deletion.

***

# Best Practices

* Prefer HTTPS for production deployments.
* Use Mutual TLS when connecting to secure enterprise services.
* Configure polling intervals based on API rate limits.
* Avoid overly aggressive polling frequencies.
* Validate authentication credentials before deployment.
* Use Retry for transient network failures.
* Monitor connector health after deployment.
* Validate payload structure before publishing to Kafka.

***

# Limitations

* Each connector monitors a single endpoint.
* Polling frequency depends on endpoint availability and response time.
* Authentication requirements depend on the external service.
* SSL certificates must remain valid for secure communication.

***

# Related Documentation

* HTTP Output Connector
* Apache Kafka Input Connector
* MQTT Input Connector
* Security
* SSL Configuration

***

# Frequently Asked Questions (FAQs)

### What is the HTTP Input Connector used for?

It ingests data from HTTP or HTTPS endpoints and publishes it to Kafka topics for downstream processing.

***

### Which input modes are supported?

The connector supports:

* Polling
* Webhook

***

### Which authentication methods are supported?

The connector supports:

* None
* Basic Authentication
* JWT
* API Key
* OAuth2
* Digest Authentication

***

### Can I connect to HTTPS endpoints?

Yes. The connector supports both HTTP and HTTPS endpoints.

***

### Does the connector support Mutual TLS?

Yes. Both One-Way SSL and Two-Way SSL (Mutual TLS) are supported.

***

### Which payload formats are supported?

Supported formats include:

* JSON
* String
* Avro
* Binary

***

### Can I periodically poll REST APIs?

Yes. Polling mode allows the connector to fetch data at configurable intervals.

***

### Can the connector receive webhook events?

Yes. Webhook mode allows external systems to push events directly into Kafka.

***

### Can I configure retry behaviour?

Yes. Retry, Skip, and Fail strategies are available for handling request failures.

***

### Can I edit a deployed connector?

Yes. Existing connector configurations can be modified and redeployed from the **Pipelines** page.

***

### Can multiple HTTP Input Connectors publish to the same Kafka topic?

Yes. Multiple connectors can publish to the same topic if required by your streaming architecture.