Article

Data Extraction using Content-Based Handles

Data Extraction using Content-Based Handles

Download full textFree with an account; signing up with Google takes seconds

Abstract

In this paper, we present an approach and a visual tool, called HWrap (Handle Based Wrapper), for creating web wrappers to extract data records from web pages. In our approach, we mainly rely on the visible page content to identify data regions on a web page. In our extraction algorithm, we inspired by the way a human user scans the page content for specific data. In particular, we use text features such as textual delimiters, keywords, constants or text patterns, which we call handles, to construct patterns for the target data regions and data records. We offer a polynomial algorithm, in which these patterns are checked against the page elements in a mixed bottom-up and top-down traverse of the DOM-tree. The extracted data is directly mapped onto a hierarchical XML structure, which forms the output of the wrapper. The wrappers that are generated by this method are robust and independent of the HTML structure. Therefore, they can be adapted to similar websites to gather and integrate information.

Authors and affiliations
  1. A. PouraminiORCID 0000-0002-3652-0935 — Department of Computer Engineering, University of Sirjan Technology, Sirjan, Iran.
  2. S. Khaje Hassani — Department of Computer Engineering, University of Sirjan Technology, Sirjan, Iran.
  3. Sh. Nasiri — Department of Computer Engineering, University of Sirjan Technology, Sirjan, Iran.

Figures (3)

  • Figure 1. A sample results page of a website for “camera”.
    Figure 1. A sample results page of a website for “camera”.page 2
  • Figure 2. Identifying handles on a web page in our proposed system, HWrap.
    Figure 2. Identifying handles on a web page in our proposed system, HWrap.page 3
  • Figure 4. Screenshot of dialog boxes for creating handles (left) and patterns (right).
    Figure 4. Screenshot of dialog boxes for creating handles (left) and patterns (right).page 5

Figures are taken from the article's full text; their rights belong to the authors and the publisher. Source: the article's original page

Similar items