Web Scraping in R with rvest

Q: rvest vs RSelenium — which should I use?

Use rvest for the vast majority of scraping: it fetches and parses static HTML and is fast, simple, and dependency-light. Reach for RSelenium (or another headless-browser tool like chromote) only when the data is rendered by JavaScript after the page loads, or when you must interact with the page — click buttons, fill forms, scroll — before the content appears. If read_html() returns the data you want, you don’t need Selenium.

Q: How do I scrape a table from a website in R?

Read the page with read_html(url), select the table with a CSS selector, and pass it to html_table(): read_html(url) |> html_element("table") |> html_table() returns a data frame with the header row as column names. If the page has several tables, use html_elements("table") |> html_table() to get a list of data frames and pick the one you need by position.

Q: What’s the difference between html_element and html_elements?

html_element() returns the first node matching your selector (a single element); html_elements() returns all matching nodes (a set). Use the singular when there’s exactly one thing to grab — a page title, or one field inside a single card — and the plural when you want a column of values or a set to map over. Mixing them up is the most common cause of an unexpectedly short or long result.

Q: Do I need dplyr to use rvest?

No. rvest’s core — read_html, html_element/html_elements, html_text2, html_attr, html_table — is self-contained. For turning repeated nodes into a tidy data frame, purrr (map() + list_rbind()) and tibble are the natural partners, and you can add dplyr afterwards to filter, mutate, or summarise the scraped data — but none of it is required to extract the data in the first place.

Parse HTML, select nodes with CSS selectors, and pull text, attributes, and tables into a data frame

Programming

A practical guide to web scraping in R with the rvest package. Read and parse HTML, select nodes with CSS selectors, pull text, attributes, and tables into a data frame, and scale up to many pages with purrr — taught the modern rvest 1.0 way (html_element/html_elements and the base pipe) on a reproducible in-memory HTML document, then applied to the real live-network patterns.

Author

Alboukadel Kassambara

Published

July 18, 2026

Modified

July 18, 2026

What you’ll learn

Read and parse HTML with rvest — read_html() for a live URL, minimal_html() for a string.
Select nodes with CSS selectors — by tag, .class, #id, and descendant — using html_element() (one) and html_elements() (many).
Extract data — clean text with html_text2(), attributes (like a link’s href) with html_attr(), and whole tables into a data frame with html_table().
Scale up with purrr — turn repeated nodes into a tidy tibble, and loop over many pages.
The modern rvest 1.0 API — html_element/html_elements and the base pipe |>, not the superseded html_node/%>%.

Need data that lives on a web page but not in a tidy download? Web scraping is the technique of extracting it programmatically — reading a page’s HTML and pulling out the parts you want. In R the standard tool is rvest (from the tidyverse), which parses HTML and lets you select and extract content with the same CSS selectors you’d use to style a page.

Most tutorials scrape a live website and paste a screenshot of the result — which nobody can reproduce, because the site changes. This guide takes a cleaner path: we teach every core mechanic on a small in-memory HTML document so every example below truly runs and returns the output you see. Then we show the real-world live-network recipes — fetching a URL, paginating, error handling — as copy-into-your-session code, clearly labelled.

Note

A note on terms. An HTML element (or node) is a tagged piece of a page — a heading <h1>, a paragraph <p>, a link <a>, a table <table>. A CSS selector is a short pattern that picks elements out of the page: table selects every table, .price selects everything with class="price", #title selects the one element with id="title". rvest uses these selectors to find what you want to extract.

Install and load rvest

Install rvest from CRAN (it comes with the tidyverse, so you may already have it). We’ll also use purrr for the scale-up section and tibble for tidy output.

install.packages(c("rvest", "purrr", "tibble"))

Then load it. Everything in the core sections needs only rvest itself:

library(rvest)

Read and parse an HTML page

Every scrape starts by parsing HTML into a document object you can query. For a live site you’d pass a URL to read_html(). To keep this guide reproducible, we build the same kind of document from a string with minimal_html() — it parses exactly like a fetched page, but the content is fixed, so the code runs anywhere.

Here is a small store page: a title, an intro line, a list of category links, and a product table. Read it once into page:

library(rvest)

page <- minimal_html('
  <html>
  <body>
    <h1 id="title">Datanovia Gadget Store</h1>
    <p class="intro">Our three best sellers this week.</p>
    <ul id="nav">
      <li><a href="https://example.com/laptops">Laptops</a></li>
      <li><a href="https://example.com/phones">Phones</a></li>
      <li><a href="https://example.com/tablets">Tablets</a></li>
    </ul>
    <table id="products">
      <tr><th>Product</th><th>Price</th><th>Rating</th></tr>
      <tr><td>Laptop</td><td>1200</td><td>4.5</td></tr>
      <tr><td>Phone</td><td>800</td><td>4.2</td></tr>
      <tr><td>Tablet</td><td>450</td><td>4.7</td></tr>
    </table>
  </body>
  </html>
')

page

{html_document}
<html>
[1] <head>\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8 ...
[2] <body>\n    <h1 id="title">Datanovia Gadget Store</h1>\n    <p class="int ...

page is a parsed document. For a real site the only change is the source:

# The live-network equivalent — fetch and parse a real page
page <- read_html("https://example.com/store")

Note

This runs it locally, not here. Any cell marked “live-network” (like read_html("https://…")) reaches out to the internet, so it can’t be part of this reproducible page — copy it into your own R session to run it. Everything built on minimal_html() executes right here.

Select one node vs many: html_element and html_elements

Two functions do the selecting, and the difference matters:

html_element() returns the first matching element (one node).
html_elements() returns all matching elements (a node set).

Pass a CSS selector to either. Pull the page title with #title (the id), then clean the text with html_text2(), which trims whitespace and collapses runs of spaces the way you’d expect:

library(rvest)
page <- minimal_html('
  <h1 id="title">Datanovia Gadget Store</h1>
  <p class="intro">Our three best sellers this week.</p>
')

page |>
  html_element("#title") |>
  html_text2()

[1] "Datanovia Gadget Store"

The base pipe |> reads left to right: take the page, select the #title element, extract its text. To grab every item in a set, switch to html_elements(). Here we pull all three category link labels at once:

library(rvest)
page <- minimal_html('
  <ul id="nav">
    <li><a href="https://example.com/laptops">Laptops</a></li>
    <li><a href="https://example.com/phones">Phones</a></li>
    <li><a href="https://example.com/tablets">Tablets</a></li>
  </ul>
')

page |>
  html_elements("a") |>
  html_text2()

[1] "Laptops" "Phones"  "Tablets"

html_element() would have returned only "Laptops". When you want a column of values, you almost always want html_elements().

Note

Modern rvest. html_element/html_elements replaced the older html_node/html_nodes in rvest 1.0.0. They behave the same; use the modern pair.

CSS selectors: tag, class, id, and descendant

A selector can target elements four common ways. Build a page with a couple of prices and a heading, then select each way:

library(rvest)
page <- minimal_html('
  <div class="product">
    <h2 class="name">Laptop</h2>
    <span class="price">1200</span>
  </div>
  <div class="product">
    <h2 class="name">Phone</h2>
    <span class="price">800</span>
  </div>
')

# By tag: every <h2> element
page |> html_elements("h2") |> html_text2()

[1] "Laptop" "Phone"

# By class: everything with class="price"
page |> html_elements(".price") |> html_text2()

[1] "1200" "800"

# Descendant: a .name that sits inside a .product
page |> html_elements(".product .name") |> html_text2()

[1] "Laptop" "Phone"

The patterns compose the same way they do in a stylesheet: .product .name (with a space) means a .name anywhere inside a .product. To find the right selector on a real site, open it in your browser, right-click the element you want, and choose Inspect — the developer tools show you the tags, classes, and ids to target.

Extract attributes: links and their hrefs

Text is only half the story. Links carry their destination in an attribute — the href. Pull attributes with html_attr(). A common task is to collect every link’s visible text alongside its URL:

library(rvest)
page <- minimal_html('
  <ul id="nav">
    <li><a href="https://example.com/laptops">Laptops</a></li>
    <li><a href="https://example.com/phones">Phones</a></li>
    <li><a href="https://example.com/tablets">Tablets</a></li>
  </ul>
')

links <- page |> html_elements("a")

data.frame(
  text = links |> html_text2(),
  url  = links |> html_attr("href")
)

     text                         url
1 Laptops https://example.com/laptops
2  Phones  https://example.com/phones
3 Tablets https://example.com/tablets

Select the <a> elements once, then extract two things from the same set — the text and the href — and bind them into a data frame. html_attr() works for any attribute: "src" on an <img>, "class", "id", or a custom data-* value.

Extract a table straight into a data frame

Many pages present data in an HTML <table>, and rvest reads one in a single call: select the table, pass it to html_table(), and you get a tibble with the header row as column names.

library(rvest)
page <- minimal_html('
  <table id="products">
    <tr><th>Product</th><th>Price</th><th>Rating</th></tr>
    <tr><td>Laptop</td><td>1200</td><td>4.5</td></tr>
    <tr><td>Phone</td><td>800</td><td>4.2</td></tr>
    <tr><td>Tablet</td><td>450</td><td>4.7</td></tr>
  </table>
')

products <- page |>
  html_element("#products") |>
  html_table()

products

# A tibble: 3 × 3
  Product Price Rating
  <chr>   <int>  <dbl>
1 Laptop   1200    4.5
2 Phone     800    4.2
3 Tablet    450    4.7

That is the whole table as a data frame — Product, Price, and Rating columns, ready to analyse. html_table() parses numeric columns as numbers, so Price and Rating come back ready for arithmetic. If a page has several tables, use html_elements("table") |> html_table() to get a list of data frames and pick the one you need.

Scale up with purrr: many nodes into a tidy tibble

Real pages repeat a structure — a grid of product cards, a list of search results. The pattern is: select the repeated container with html_elements(), then purrr::map() over each one to pull its fields. This turns a page of cards into one tidy tibble.

Here three product cards each hold a name, a price, and a rating. We select the cards, map a small extractor over each with purrr::map(), then stack the per-card rows into one data frame with purrr::list_rbind() — the current purrr idiom, which replaced the now-superseded map_dfr() in purrr 1.0.0:

library(rvest)
library(purrr)
library(tibble)

page <- minimal_html('
  <div class="product">
    <h2 class="name">Laptop</h2>
    <span class="price">1200</span>
    <span class="rating">4.5</span>
  </div>
  <div class="product">
    <h2 class="name">Phone</h2>
    <span class="price">800</span>
    <span class="rating">4.2</span>
  </div>
  <div class="product">
    <h2 class="name">Tablet</h2>
    <span class="price">450</span>
    <span class="rating">4.7</span>
  </div>
')

cards <- page |> html_elements(".product")

products <- cards |>
  map(function(card) {
    tibble(
      name   = card |> html_element(".name")   |> html_text2(),
      price  = card |> html_element(".price")  |> html_text2() |> as.numeric(),
      rating = card |> html_element(".rating") |> html_text2() |> as.numeric()
    )
  }) |>
  list_rbind()

products

# A tibble: 3 × 3
  name   price rating
  <chr>  <dbl>  <dbl>
1 Laptop  1200    4.5
2 Phone    800    4.2
3 Tablet   450    4.7

Inside the function, note the switch back to html_element() (singular) — each card has exactly one name, price, and rating, so you want the first (and only) match per card. Selecting from card rather than page keeps each row’s fields together. This “select the set, map over it” pattern is the workhorse of any non-trivial scrape.

A bar chart of three product ratings — Laptop 4.5, Phone 4.2, Tablet 4.7 — extracted from an HTML page into a data frame with rvest in R.

Once the data is a tibble, it flows straight into the rest of your R workflow — here the three scraped ratings, plotted the moment they leave the page.

Loop over many pages

Paginated sites split results across ?page=1, ?page=2, and so on. The scale-up pattern extends cleanly: map over the page numbers, fetch and parse each, and stack the results. Because this hits a live server, it is a run-it-locally recipe — but the shape is exactly the reproducible one above, with read_html() swapped in for minimal_html():

library(rvest)
library(purrr)
library(tibble)

scrape_page <- function(page_number) {
  url <- paste0("https://example.com/products?page=", page_number)
  page <- read_html(url)

  page |>
    html_elements(".product") |>
    map(function(card) {
      tibble(
        name  = card |> html_element(".name")  |> html_text2(),
        price = card |> html_element(".price") |> html_text2()
      )
    }) |>
    list_rbind()
}

# Fetch pages 1 to 5 and combine into one tibble
all_products <- map(1:5, function(p) {
  result <- scrape_page(p)
  Sys.sleep(1)   # pause 1s between requests — be polite to the server
  result
}) |> list_rbind()

Sys.sleep(1) pauses a second between requests so you don’t hammer the server — a basic courtesy that also keeps you from getting rate-limited or blocked.

Sessions and error handling

Two more live-network patterns worth knowing. A session keeps state (cookies, headers) across requests to the same site, which some pages require — start one with session() and move around with session_jump_to():

# A browsing session that persists cookies/headers
s <- session("https://example.com")
s <- s |> session_jump_to("https://example.com/products")

s |> html_elements(".product .name") |> html_text2()

And because a live fetch can fail — a 404, a timeout, a blocked request — wrap read_html() in tryCatch() so one bad page doesn’t crash a long scrape:

safe_read <- function(url) {
  tryCatch(
    read_html(url),
    error = function(e) {
      message("Skipping ", url, ": ", conditionMessage(e))
      NULL   # return NULL instead of stopping
    }
  )
}

page <- safe_read("https://example.com/does-not-exist")
if (is.null(page)) message("No page — moving on.")

Returning NULL on failure lets your map() loop skip the bad page and keep going. Combine tryCatch() with Sys.sleep() and you have the backbone of a robust, polite scraper.

Scrape politely and legally

Before you scrape a site, check what it allows. Most sites publish a robots.txt file (at https://site.com/robots.txt) stating which paths automated clients may fetch, and their terms of service may restrict scraping or reuse of the data. A few habits keep you on the right side:

Read robots.txt and the terms of service. The polite package can check robots.txt for you and manage rate limits.
Rate-limit yourself with Sys.sleep() between requests, as above.
Prefer an official API when one exists — it’s faster, more stable, and clearly permitted.
Cache what you fetch so you don’t re-download the same page while developing.

Important

This is general guidance, not legal advice. Scraping law and a site’s terms vary by jurisdiction and by site — when in doubt about a specific site or dataset, check its terms and seek qualified advice.

Common issues

html_element() returns NA or nothing. Almost always the CSS selector doesn’t match. Open the page in your browser’s Inspect tool and confirm the exact tag, class, or id — a class you assumed was .price might be .product-price. Remember html_element() returns only the first match; if you expected several values, use html_elements().

You get one value when you wanted many (or vice-versa). This is the html_element vs html_elements distinction. Singular returns the first node; plural returns the whole set. For a data-frame column, use the plural.

read_html() returns a login page or an almost-empty document. The content is likely rendered by JavaScript after the page loads, so it isn’t in the initial HTML rvest fetches. rvest reads static HTML only — for JavaScript-heavy sites you need a headless browser (see the RSelenium question below), or check whether the site exposes an API you can call directly.

Frequently asked questions

Is web scraping legal in R?

Web scraping itself is a neutral technique — the legality depends on what you scrape and how, not on the language. Public, factual data fetched politely is generally lower-risk; scraping personal data, copyrighted content, or paths a site’s robots.txt or terms of service forbid is not. Always check the site’s robots.txt and terms, rate-limit your requests, and prefer an official API where one exists. This is general information, not legal advice — rules vary by jurisdiction and by site.

rvest vs RSelenium — which should I use?

Use rvest for the vast majority of scraping: it fetches and parses static HTML and is fast, simple, and dependency-light. Reach for RSelenium (or another headless-browser tool like chromote) only when the data is rendered by JavaScript after the page loads, or when you must interact with the page — click buttons, fill forms, scroll — before the content appears. If read_html() returns the data you want, you don’t need Selenium.

How do I scrape a table from a website in R?

Read the page with read_html(url), select the table with a CSS selector, and pass it to html_table(): read_html(url) |> html_element("table") |> html_table() returns a data frame with the header row as column names. If the page has several tables, use html_elements("table") |> html_table() to get a list of data frames and pick the one you need by position.

What’s the difference between html_element and html_elements?

html_element() returns the first node matching your selector (a single element); html_elements() returns all matching nodes (a set). Use the singular when there’s exactly one thing to grab — a page title, or one field inside a single card — and the plural when you want a column of values or a set to map over. Mixing them up is the most common cause of an unexpectedly short or long result.

Do I need dplyr to use rvest?

No. rvest’s core — read_html, html_element/html_elements, html_text2, html_attr, html_table — is self-contained. For turning repeated nodes into a tidy data frame, purrr (map() + list_rbind()) and tibble are the natural partners, and you can add dplyr afterwards to filter, mutate, or summarise the scraped data — but none of it is required to extract the data in the first place.

Test your understanding

Your turn: extract a tidy tibble

You’re given this in-memory page. Select the three book cards and build a tibble with a title column and a numeric year column.

library(rvest)
library(purrr)
library(tibble)

page <- minimal_html('
  <div class="book"><h3 class="title">R Graphics</h3><span class="year">2019</span></div>
  <div class="book"><h3 class="title">Practical Guide</h3><span class="year">2017</span></div>
  <div class="book"><h3 class="title">ggplot2 Essentials</h3><span class="year">2022</span></div>
')

Hint

Select the repeated container with html_elements(".book"), then map() over the set and list_rbind() the rows. Inside the function, use html_element() (singular) for .title and .year, and wrap the year in as.numeric().

Solution

books <- page |>
  html_elements(".book") |>
  map(function(card) {
    tibble(
      title = card |> html_element(".title") |> html_text2(),
      year  = card |> html_element(".year")  |> html_text2() |> as.numeric()
    )
  }) |>
  list_rbind()

books
#> # A tibble: 3 x 2
#>   title               year
#>   <chr>              <dbl>
#> 1 R Graphics          2019
#> 2 Practical Guide     2017
#> 3 ggplot2 Essentials  2022

Select the set, map over it, pull one field per card with the singular html_element() — the same pattern as the product cards above.

Quick check. You run page |> html_element(".price") |> html_text2() on a page with five .price elements. How many values come back?

Show answer

One — the text of the first .price element. html_element() (singular) always returns a single node. To get all five, use html_elements() (plural).

Conclusion

rvest turns a web page into data with a handful of composable verbs: read_html() (or minimal_html()) to parse, html_element/html_elements() to select with CSS selectors, and html_text2(), html_attr(), and html_table() to extract text, attributes, and tables. Add purrr::map() + list_rbind() to fold repeated nodes and many pages into one tidy tibble, wrap live fetches in tryCatch() and Sys.sleep(), and check robots.txt before you start. Because we taught the mechanics on an in-memory document, every core example here is copy-paste reproducible — swap minimal_html() for read_html(url) and the same code scrapes the real web.

Citation

BibTeX citation:

@online{kassambara2026,
  author = {Kassambara, Alboukadel},
  title = {Web {Scraping} in {R} with Rvest},
  date = {2026-07-18},
  url = {https://www.datanovia.com/blog/web-scraping-in-r-with-rvest},
  langid = {en}
}

For attribution, please cite this work as:

Kassambara, Alboukadel. 2026. “Web Scraping in R with Rvest.” July 18. https://www.datanovia.com/blog/web-scraping-in-r-with-rvest.

Install and load rvest

Read and parse an HTML page

Select one node vs many: html_element and html_elements

CSS selectors: tag, class, id, and descendant

Extract attributes: links and their hrefs

Extract a table straight into a data frame

Scale up with purrr: many nodes into a tidy tibble

Loop over many pages

Sessions and error handling

Scrape politely and legally

Common issues

Frequently asked questions

Test your understanding

Conclusion

Related

Citation