Skip to main content
The X API uses pagination for endpoints that return multiple pages of results (e.g. timelines, search etc.). Each API call response includes a meta object with result_count, previous_token, and next_token. The XDK takes care of making multiple API calls using the next_token so developers can just specify how much data they are looking for without having to make multiple calls. The SDK simplifies this with:
  • Built-in Iterators: Use generator functions for seamless multi-page fetching.
  • Explicit Token Handling: For flexible manual control when needed by passing pagination_token when needed.
  • Max Results Enforcement: Respect max_results per call (up to API limits, e.g., 100 for search).
Use the iterate() method on paginated responses to fetch all results lazily. Example: Paginated Search 
from xdk import Client

client = Client(bearer_token="your_bearer_token")

# Search with automatic pagination
all_posts = []
for page in client.posts.search_recent(
    query="python",
    max_results=100,  # Per page
    tweetfields=["created_at", "author_id"]  # Optional expansions
):
    all_posts.extend(page.data)
    print(f"Fetched {len(page.data)} Posts (total: {len(all_posts)})")

print(f"Total tweets: {len(all_posts)}")
  • The iterator handles next_token automatically.
  • Stops when no next_token is present.
  • Supports rate limit backoff to avoid 429 errors.

​
Manual Pagination

If you require control over the results for some custom logic (e.g. processing page-by-page), you can still use the next_token and do the pagination manually as shown below: 
response = client.posts.search_recent(
    query="xdk python sdk",
    max_results=100,
    pagination_token=None  # First page
)

print(f"First page: {len(response.data)} Posts")
next_token = response.meta.next_token

if next_token:
    next_response = client.posts.search_recent(
        query="xdk python sdk",
        max_results=100,
        pagination_token=next_token
    )
    print(f"Second page: {len(next_response.data)} Posts")
Tips:
  • Always specify max_results to optimize (default varies by endpoint).
  • Monitor meta.result_count for debugging.
  • For very large queries, consider async iteration to avoid blocking.