Class: Heatwave::Crawler::Extract::Seo

Inherits:
Object
  • Object
show all
Defined in:
app/services/heatwave/crawler/extract/seo.rb

Overview

Technical-SEO field extraction from an HTML document. Used by
:inspect mode against the rendered DOM (catches JS-injected schema
and meta that raw-HTML parsing misses) and shared with
Cache::SiteCrawler for JSON-LD extraction.

Class Method Summary collapse

Class Method Details

.call(html) ⇒ Hash

Returns title, meta_description, canonical, robots,
hreflang, h1s, schemas.

Parameters:

  • html (String) —

    HTML (rendered or raw)

Returns:

  • (Hash) —

    title, meta_description, canonical, robots,
    hreflang, h1s, schemas



14
15
16
17
18
19
20
21
22
23
24
25
26
27
 
# File 'app/services/heatwave/crawler/extract/seo.rb', line 14

def call(html)
  doc = Nokogiri::HTML(html)
  {
    title: doc.at_css('title')&.text&.strip,
    meta_description: doc.at_css('meta[name="description"]')&.[]('content')&.strip,
    canonical: doc.at_css('link[rel="canonical"]')&.[]('href'),
    robots: doc.at_css('meta[name="robots"]')&.[]('content'),
    hreflang: doc.css('link[rel="alternate"][hreflang]').map do |link|
      { lang: link['hreflang'], href: link['href'] }
    end,
    h1s: doc.css('h1').map { |h| h.text.strip }.compact_blank,
    schemas: extract_json_ld_schemas(html)
  }
end

.extract_json_ld_schemas(html) ⇒ Array<Hash>

Extract all JSON-LD structured data blocks from HTML.
Parses tags and returns an array
of schema objects. Handles @graph containers by flattening nested
schemas.

Parameters:

  • html (String) —

    HTML content

Returns:

  • (Array<Hash>) —

    parsed JSON-LD schema objects (empty array if none found)



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'app/services/heatwave/crawler/extract/seo.rb', line 36

def extract_json_ld_schemas(html)
  doc = Nokogiri::HTML(html)
  schemas = []

  doc.css('script[type="application/ld+json"]').each do |script|
    raw = script.text.strip
    next if raw.blank?

    parsed = JSON.parse(raw)

    # Flatten @graph containers so each schema type is a top-level entry
    case parsed
    when Hash
      if parsed['@graph']
        Array(parsed['@graph']).each { |item| schemas << item if item.is_a?(Hash) }
      else
        schemas << parsed
      end
    when Array
      parsed.each { |item| schemas << item if item.is_a?(Hash) }
    end
  rescue JSON::ParserError => e
    Rails.logger.warn "[Crawler::Extract::Seo] Invalid JSON-LD on page: #{e.message}"
  end

  schemas
end