Class PuppeteerWebBaseLoader

Class that extends the BaseDocumentLoader class and implements the DocumentLoader interface. It represents a document loader for scraping web pages using Puppeteer.

Example

const loader = new PuppeteerWebBaseLoader("https:exampleurl.com", {
  launchOptions: {
    headless: true,
  },
  gotoOptions: {
    waitUntil: "domcontentloaded",
  },
});
const screenshot = await loader.screenshot();

Hierarchy

Implements

Constructors

constructor

Properties

options webPath

Methods

load loadAndSplit scrape screenshot imports

Constructors

constructor

Properties

options

options: undefined | PuppeteerWebBaseLoaderOptions

webPath

webPath: string

Methods

load

loadAndSplit

scrape

  • Method that calls the _scrape method to perform the scraping of the web page specified by the webPath property.

    Returns Promise<string>

    Promise that resolves to the scraped HTML content of the web page.

screenshot

  • Screenshot a web page and return it as a Toolkit object where the pageContent property is the screenshot encoded in base64.

    Returns Promise<Document>

    A document object containing the screenshot of the page encoded in base64.

Static imports

  • Static method that imports the necessary Puppeteer modules. It returns a Promise that resolves to an object containing the imported modules.

    Returns Promise<{
        launch: ((options?) => Promise<Browser>);
    }>

    Promise that resolves to an object containing the imported Puppeteer modules.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc