How to Scrape Tweets (2026)

If you want to scrape tweets in 2026, the useful news is that a single public tweet still gives up its text, its engagement metrics, its media, and its author without a login. The catch is that the plain routes from the old tutorials are gone. X put timelines and search behind an auth wall in 2023 and now ships defensive changes to its web app every few weeks, so a naive request to a status URL returns a login shell instead of the tweet.

This guide is about the tweet object itself, not a broad platform overview. I will show you exactly what one tweet exposes, the Python that pulls a single tweet by id, how to reconstruct a whole reply thread, what triggers a block, and the managed route I reach for when I need volume. Every field and call shape below is one I checked against live X targets in July 2026.

Can you still scrape tweets in 2026?

Yes, you can still scrape tweets in 2026, and a single public tweet is the most reliable thing to pull. X’s own web client renders a tweet by calling internal endpoints that accept a short-lived guest token, so a logged-out request can reach the same data the site shows a visitor. What comes back is the full structured tweet, not the HTML shell around it.

What stays out of reach without an account is narrow and worth stating up front. You cannot scrape protected accounts, direct messages, or a full follower list while logged out, and X caps most of the search and reply timeline behind authentication. Public tweets, their visible replies, and their media metadata are the reachable surface.

The official alternative is the X API, and its economics are the reason most projects scrape the front end instead. Per the official X API documentation, v2 now runs on pay-per-usage pricing where credits are deducted per request and the same resource requested twice in 24 hours is charged once. Reading posts is metered per resource, so a research pull of hundreds of thousands of tweets climbs fast. Before you request anything, it helps to know exactly which fields a tweet hands back.

What data does a single tweet contain?

A single tweet contains four groups of data worth scraping: the text and metadata, the engagement metrics, the media, and the author. When you scrape a tweet by id from X’s internal endpoint, all four arrive in one JSON object, and a complete scraper keeps every group intact.

• Text and metadata: the tweet text (full_text), the tweet id and URL, the created_at timestamp, the detected lang, and the entities (hashtags, mentions, and expanded links) parsed out of the body.
• Engagement metrics: favorite_count (likes), retweet_count (reposts), reply_count, quote_count, bookmark_count, and a view count where X includes it. These are the numbers that turn a tweet into a data point for reach and sentiment work.
• Media: for a tweet with attachments, the media entities carry media_url_https, the type (photo, video, or animated_gif), and the size variants. A video tweet adds a video_info block with bitrate-ranked variants you can download.
• Author: a nested user object with the screen_name (handle), display name, numeric user id, followers_count, and verified status, so you know who posted and how large their audience is.

The view count is the one field to treat as optional. X serves it from a separate views object that is not always present on a logged-out fetch, so any scraper that promises it every time is guessing. With the fields defined, the next step is requesting them in code.

How do you scrape a single tweet with Python?

You scrape a single tweet with Python by activating a guest token, then calling X’s internal TweetResultByRestId GraphQL operation with the tweet’s numeric id, passing the token in an x-guest-token header. The guest token is the key that unlocks the read endpoints the website itself uses, and it comes back in one request.

Here is the activation step, which returned 200 and a fresh token every time I ran it in July 2026:

import requests

# The public web-client bearer X ships in its own JavaScript.
BEARER = ("AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs"
          "%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA")

r = requests.post(
    "https://api.x.com/1.1/guest/activate.json",
    headers={"Authorization": f"Bearer {BEARER}"},
    timeout=15,
)
guest_token = r.json()["guest_token"]
print(r.status_code, guest_token)   # -> 200 20668...

With that token, you call the TweetResultByRestId operation. Each GraphQL query has an opaque doc_id that X rotates every few weeks, so you capture the current one from the site’s network traffic rather than hardcoding it for good:

DOC_ID = "PASTE_CURRENT_DOC_ID"   # rotates; grab it from the browser network tab
tweet_id = "1234567890123456789"

resp = requests.get(
    f"https://api.x.com/graphql/{DOC_ID}/TweetResultByRestId",
    headers={
        "Authorization": f"Bearer {BEARER}",
        "x-guest-token": guest_token,
    },
    params={"variables": '{"tweetId":"%s","withCommunity":false}' % tweet_id},
    timeout=20,
)
result = resp.json()["data"]["tweetResult"]["result"]
legacy = result["legacy"]
print(legacy["full_text"], "-", legacy["favorite_count"], "likes")

The legacy block holds full_text, favorite_count, retweet_count, and the media entities, while the sibling core block holds the author. This works today, but the guest token expires within hours and the doc_id moves, so a durable version re-activates tokens and re-captures ids on a schedule. My full Python walkthrough, including the cookie-auth libraries that replay the same API with a real account’s session, is in how to scrape Twitter with Python. Once you can pull one tweet, the next question is the replies underneath it.

How do you scrape a tweet’s replies and full thread?

You scrape a tweet’s replies and full thread by calling X’s TweetDetail GraphQL operation instead of TweetResultByRestId, because TweetDetail returns the conversation timeline: the original tweet plus the replies X renders beneath it. You pass the same guest token and the tweet id, and the response is an ordered list of timeline entries you walk to collect each reply.

Rebuilding the thread into a tree relies on one field. Every reply carries an in_reply_to_status_id (the id of the tweet it answers) and shares a single conversation_id with the whole thread, where that conversation id equals the root tweet’s id. So you can parse a flat list of replies and reconstruct the parent-child structure yourself:

# `replies` is the flat list of tweet objects parsed from the TweetDetail entries.
by_parent = {}
for t in replies:
    parent = t.get("in_reply_to_status_id_str")
    by_parent.setdefault(parent, []).append(t)

def render(status_id, depth=0):
    for child in by_parent.get(status_id, []):
        print("  " * depth, child["full_text"][:80])
        render(child["id_str"], depth + 1)   # recurse into replies-to-replies

render(root_tweet_id)

The limit here is coverage, not code. X heavily walls the reply timeline for logged-out clients, so a guest token returns the top slice of a conversation and paginates poorly past it. Deep threads, or ones on rate-limited accounts, usually need a logged-in session or a managed scraper to capture in full. That coverage ceiling is one symptom of a broader problem: the fetch layer, not the parsing, is where tweet scrapers actually fail.

How do you scrape tweets without getting blocked?

You scrape tweets without getting blocked by staying off datacenter IPs, keeping request volume low per IP, and refreshing the guest token before it expires. X challenges a datacenter address within a request or two and caps anonymous access at roughly 300 requests per hour, so a scraper running on a plain cloud server fails before it reads a single tweet.

X also runs active browser challenges on its gated endpoints. It uses Cloudflare Turnstile fingerprinting on login walls and rate-limited routes, which passively scores whether a client looks like a real browser and quietly fails naive scripts. A bare HTTP request from a datacenter trips this where a real user never notices it.

The checklist that kept me collecting is short:

• Route through residential IPs, not the datacenter ranges X pre-flags.
• Re-activate the guest token every couple of hours, before it expires.
• Add delays between requests and stay well under the ~300/hour anonymous ceiling.
• Re-capture the doc_id values when a working call suddenly starts returning errors.

Doing all of that yourself is a standing maintenance project. I break the block triggers down in full in scraping Twitter without getting blocked. Past a few hundred tweets, that upkeep is the reason most teams hand the proxy-and-token problem to an API.

How do you scrape tweets at scale without managing tokens?

You scrape tweets at scale without managing tokens by sending the tweet id to a managed scraper API and letting it return parsed JSON, with the guest-token rotation, doc_id capture, and residential proxy pool handled on the server side. You send one authenticated request and get the tweet object back, so none of the breakage from the previous section lands on you.

This is the route I reach for on scheduled collection. A single tweet call is a plain GET with your API key as a query parameter:

curl "https://chocodata.com/api/v1/twitter/tweet?id=1234567890&api_key=$CHOCO_API_KEY"

The Python version drops straight into a data pipeline and returns the same text, metrics, media, and author fields you would otherwise parse out of a GraphQL payload:

import requests

resp = requests.get(
    "https://chocodata.com/api/v1/twitter/tweet",
    params={"id": "1234567890", "api_key": "YOUR_CHOCO_API_KEY"},
    timeout=30,
)
resp.raise_for_status()
tweet = resp.json()["data"]

print(tweet["text"], "-", tweet["favorite_count"], "likes")
print(tweet["author"]["screen_name"], "-", tweet["author"]["followers_count"], "followers")

When I sent that request with a deliberately invalid key in July 2026, ChocoData returned 401 {"error":{"code":"INVALID_API_KEY","message":"Api key not recognised."}}, which confirms the auth step is real and the endpoint is live. A keyword or hashtag pull uses the same key against the search resource by swapping the path to /api/v1/twitter/search?query=..., so one integration covers single tweets and search. The free tier covers 1,000 requests to start, and if you are weighing providers I ranked the managed options in the best Twitter scrapers of 2026. Whichever route you pick, the last thing to settle is where the legal line sits.

Is it legal to scrape tweets?

Scraping publicly visible tweets while logged out sits on favorable US legal ground, but X’s own terms add contractual risk that scraping public web data generally does not. The law and the contract point in different directions, so it helps to separate them.

On the legal side, US courts have repeatedly protected logged-off scraping of public data. In Meta Platforms v. Bright Data, decided in January 2024, the court granted summary judgment for Bright Data and held that a platform’s terms do not bar logged-off scraping of public data. That built on the earlier hiQ v. LinkedIn line, where the Ninth Circuit found that scraping public data does not violate the Computer Fraud and Abuse Act. Data you can see without an account is the data courts have been least willing to fence off.

On the contract side, X is stricter than most. Its Terms of Service set liquidated damages of $15,000 for any party that accesses more than 1,000,000 posts in a 24-hour period through automated means without permission. That does not make scraping a single public tweet a criminal act, but logged-in scraping, very large pulls, and reselling data carry real exposure. Scope your collection to public tweets, keep the volume reasonable, and the tweet-level methods above stay on the safe side of both the law and the terms.