Skip to main content

Documentation Index

Fetch the complete documentation index at: https://documentation.byteful.com/llms.txt

Use this file to discover all available pages before exploring further.

The Byteful API provides comprehensive analytics and usage tracking capabilities to help you monitor and optimize your proxy infrastructure. This guide explains the different components of the tracking system, how they work together, and how to access them through the API.

​
Analytics Components

Byteful employs multiple complementary systems to track and analyze proxy usage:
  • Raw Logs (log): Detailed individual request data, retained for 7 days
  • Log Summaries (log_summary): Aggregated daily usage patterns, retained for 90+ days
  • Residential Ledger (residential_ledger): Residential data accounting records, retained indefinitely
  • Mobile Ledger (mobile_ledger): Mobile data accounting records, retained indefinitely
  • Analytics Graphs: Visualizations generated from logs and summaries, with indefinite historical data access
The /analytics/graph endpoint processes data from both logs and log summaries to provide comprehensive visualizations regardless of the age of the data being analyzed.

​
Log Objects

​
Raw Logs (log)

Raw logs represent individual proxy requests and provide the most detailed information. They are stored for 7 days.
  • Tracked at the proxy_user_id level
  • Created for every single proxy request
  • Contains detailed information such as:
    • Client IP address
    • Request size in bytes
    • HTTP status/error codes
    • Precise request datetime
    • Authentication type
    • Hostname being accessed
    • Geographic information (country, city)
    • ASN information
Example raw log object: 
{
  "log_id": "123e4567-e89b-12d3-a456-426614174000",
  "proxy_user_id": "stevejobs",
  "log_network": "isp",
  "log_protocol": "http",
  "log_hostname": "apple.com",
  "log_client_ip_address": "17.172.224.1",
  "log_total_bytes": 5120,
  "log_request_datetime": "2025-04-01 13:00:00",
  "country_id": "us",
  "city_alias": "cupertino",
  "asn_id": 1299
}

​
Log Summaries (log_summary)

Log summaries aggregate raw logs into daily summaries. They provide an efficient way to analyze usage patterns without storing every individual request.
  • Initially organized by proxy_user_id, network, and hostname for the first 90 days
  • After 90 days, further consolidated to just proxy_user_id and network level (hostname details are removed)
  • Includes metrics like:
    • Total requests
    • Total bytes transferred
    • Success/error counts
Example log summary object: 
{
  "log_summary_id": "456e7890-e89b-12d3-a456-426614174000",
  "proxy_user_id": "stevejobs",
  "log_summary_network": "residential",
  "log_summary_hostname": "apple.com",
  "log_summary_requests": 100,
  "log_summary_bytes": 10000,
  "log_summary_period": "2025-04-01 00:00:00"
}

​
Residential Ledger (residential_ledger)

The residential ledger specifically tracks data usage for residential proxies, which operate on a data-based billing model rather than a per-proxy model.
  • Tracks daily residential data usage at the customer account level
  • Records various types of data changes:
    • Usage (decrements)
    • Top-ups (increments when purchasing additional data)
    • Service purchases (addition of data with new residential services)
    • Refunds and adjustments (manual or automatic credits)
  • Provides a complete audit trail of all changes to your residential data allocation
  • Critical for billing and quota management
Example residential ledger object: 
{
  "residential_ledger_id": "123e4567-e89b-12d3-a456-426614174000",
  "residential_ledger_bytes": 128290101,
  "residential_ledger_requests": 1244,
  "residential_ledger_period_date": "2025-04-01",
  "residential_ledger_reason": "usage"
}

​
Mobile Ledger (mobile_ledger)

The mobile ledger specifically tracks data usage for mobile proxies, which operate on a data-based billing model rather than a per-proxy model.
  • Tracks daily mobile data usage at the customer account level
  • Records various types of data changes:
    • Usage (decrements)
    • Top-ups (increments when purchasing additional data)
    • Service purchases (addition of data with new mobile services)
    • Refunds and adjustments (manual or automatic credits)
  • Provides a complete audit trail of all changes to your mobile data allocation
  • Critical for billing and quota management
Example mobile ledger object: 
{
  "mobile_ledger_id": "123e4567-e89b-12d3-a456-426614174000",
  "mobile_ledger_bytes": 128290101,
  "mobile_ledger_requests": 1244,
  "mobile_ledger_period_date": "2025-04-01",
  "mobile_ledger_reason": "usage"
}

​
Data Flow Process

The tracking system follows specific data flows depending on the proxy type:

​
Datacenter and ISP Proxy Requests

  1. When a request is made through a datacenter or ISP proxy:
    • A log record is created
    • The corresponding log_summary is incremented or created if it doesn’t exist

​
Residential Proxy Requests

Residential proxies involve an additional tracking layer due to their data-based billing model:
  1. When a request is made through a residential proxy:
    • A log record is created
    • The corresponding log_summary is incremented or created
    • The customer’s residential_ledger usage record is incremented or created

​
Mobile Proxy Requests

Mobile proxies involve an additional tracking layer due to their data-based billing model:
  1. When a request is made through a mobile proxy:
    • A log record is created
    • The corresponding log_summary is incremented or created
    • The customer’s mobile_ledger usage record is incremented or created

​
Accessing Analytics Data

​
Residential & Mobile Ledger Endpoints

To monitor residential data usage: 
GET /public/user/residential_ledger/search
GET /public/user/residential_ledger/retrieve/{residential_ledger_id}
These endpoints allow you to track daily customer usage and top-ups for residential proxies. To monitor mobile data usage: 
GET /public/user/mobile_ledger/search
GET /public/user/mobile_ledger/retrieve/{mobile_ledger_id}
These endpoints allow you to track daily customer usage and top-ups for mobile proxies.

​
Analytics Graph Endpoint

For visualizing usage patterns and trends: 
GET /public/user/analytics/graph
This powerful endpoint:
  • Automatically analyzes logs and log summary records
  • Produces summaries and graphing data over specified time periods
  • Supports filtering by proxy user, network, and hostname
  • Offers flexible time intervals (minute, hour, day, month)

​
Raw Log Endpoints

For detailed troubleshooting within the 7-day retention window: 
GET /public/user/log/search
GET /public/user/log/retrieve/{log_id}

​
Log Summary Endpoints

For aggregated historical data analysis: 
GET /public/user/log_summary/search
GET /public/user/log_summary/retrieve/{log_summary_id}

​
Endpoint Selection Guide

Use /analytics/graph for most monitoring needs. Use /log or /log_summary for detailed breakdowns by hostname, proxy user, or network. Use /residential_ledger for residential data tracking and /mobile_ledger for mobile data tracking.

​
Reference Table: Analytics Endpoints

EndpointPurposeRetentionGranularity
/log/searchSearch raw logs7 daysIndividual requests
/log/retrieve/{log_id}Get specific raw log7 daysIndividual request
/log_summary/searchSearch log summaries90+ daysDaily aggregations
/log_summary/retrieve/{log_summary_id}Get specific log summary90+ daysDaily aggregation
/residential_ledger/searchSearch residential usage recordsIndefiniteDaily usage
/residential_ledger/retrieve/{residential_ledger_id}Get specific residential usage entryIndefiniteDaily usage
/mobile_ledger/searchSearch mobile usage recordsIndefiniteDaily usage
/mobile_ledger/retrieve/{mobile_ledger_id}Get specific mobile usage entryIndefiniteDaily usage
/analytics/graphVisualize usage patternsIndefiniteConfigurable intervals
/residential/summaryGet overall residential data statusCurrentAccount-level summary
/mobile/summaryGet overall mobile data statusCurrentAccount-level summary