Laravel sitemap generator and crawler package for SEO optimization. Supports multilingual sites, image/video indexing, link validation, priority scoring, and advanced crawling automation.
anassrojea/laracrawler is a Laravel package for laravel sitemap generator and crawler package for seo optimization. supports multilingual sites, image/video indexing, link validation, priority scoring, and advanced crawling automation..
It currently has 0 GitHub stars and 89 downloads on Packagist (latest version 1.2.0).
Install it with composer require anassrojea/laracrawler.
Discover more Laravel packages by anassrojea
or browse all Laravel packages to compare alternatives.
Last updated
A powerful Laravel sitemap generator with crawling, validation, multilingual support, priority auto-scoring, indexability audit, and more.
Optimized for Google SEO best practices.
hreflang) with validation<img> + <picture> sources<image:title> and <image:caption> from alt/title<video>, <source>, and <iframe> (YouTube, Vimeo)<video:title> and <video:description> (defaults configurable)priority_boostnow → always current timefile → file modification timedb → fetch from database columncallback → resolve dynamically via Closure/servicenoindex in headers (X-Robots-Tag) or meta tagssitemap-errors.xmlsitemap-errors.xml50k URLs or 50MB limit)sitemap-index.xmlcomposer require anassrojea/laracrawler
Publish config:
php artisan vendor:publish --tag=laracrawler-config
Generate sitemap:
php artisan laracrawler:generate
Options:
--summary → Print summary of exclusions.--debug → Extra debug output.--validate → Force validation of links even if disabled in config.config/sitemap.php)'base_url' => env('APP_URL', 'https://example.com'),
'xdefault' => 'https://example.com', // <xhtml:link hreflang="x-default">
'validate_links' => false,
'max_errors' => 5000,
'exclude_urls' => [
'/admin',
'#\?page=\d+#', // regex pagination
'#/search#',
'#\.(css|js)$#',
],
'exclude_assets' => [
'#\.(css|js|json|xml|txt|md)$#',
'#\.(zip|rar|tar|gz|7z)$#',
],
'normalize' => [
'strip_queries' => true,
'strip_anchors' => true,
'strip_trailing_slash'=> true,
'canonicalize' => true, // lowercase
'enforce_https' => true,
'enforce_www' => null, // true = add, false = strip
'force_trailing_slash'=> false,
],
'default_lang' => 'en',
'lang_mode' => 'path', // "path", "subdomain", or "query"
'alternates' => [
'en' => 'https://example.com/en',
'ar' => 'https://example.com/ar',
'tr' => 'https://example.com/tr',
],
'include' => [
'urls' => true,
'images' => true,
'videos' => true,
'languages' => true,
'rules' => [
'#/blog#' => [
'images' => true,
'videos' => false,
],
],
],
'image_whitelist' => [
// '/storage/uploads/services/',
],
'image_defaults' => [
'title' => 'Image Title',
'description' => 'Image Description',
],
'video_whitelist' => [
// '/storage/uploads/services/',
],
'video_defaults' => [
'title' => 'Video Title',
'description' => 'Video Description',
],
Rules let you override defaults per URL pattern:
'rules' => [
'/$' => [ // homepage
'changefreq' => 'daily',
'priority' => '1.0',
'lastmod' => 'now',
],
'/blog' => [
'changefreq' => 'daily',
'priority' => '0.9',
'priority_boost'=> 0.3, // 🚀 boost blogs slightly
'lastmod' => [
'strategy' => 'db',
'table' => 'posts',
'lookup' => 'slug',
'column' => 'updated_at',
],
],
'#^/(en|ar|tr)?/service#' => [
'changefreq' => 'weekly',
'priority' => null, // auto-score
'priority_boost'=> 0.3, // 🚀 boost services
'lastmod' => [
'strategy' => 'db',
'table' => 'services',
'lookup' => 'slug',
'column' => 'updated_at',
],
],
],
priority → fixed value (0.1–1.0) or null for auto-score.priority_boost → bump score (applied only if auto-score).lastmod strategies:
"now" → always current timestamp"file" → filesystem mtime"db" → fetch updated_at from DB"callback" → custom closure or service'priority_scoring' => [
'enabled' => true,
'weights' => [
'depth' => 0.4,
'links' => 0.4,
'freshness' => 0.2,
],
'min' => 0.1,
'max' => 1.0,
],
'ping' => true,
'ping_targets' => [
'Google' => 'http://www.google.com/ping?sitemap=',
'Bing' => 'http://www.bing.com/ping?sitemap=',
'Yandex' => 'https://webmaster.yandex.com/ping?sitemap=',
'Baidu' => 'http://ping.baidu.com/ping?sitemap=',
],
'queue' => [
'enabled' => false,
'connection' => 'default',
'batch_size' => 100,
],
'http' => [
'validate_links' => [
'timeout' => 10,
'connect_timeout' => 5,
'verify' => false,
'http_errors' => false,
'headers' => [
'User-Agent' => 'LaracrawlerBot/1.0 (https://example.com)',
],
],
'validate_alternates' => [
'timeout' => 5,
'connect_timeout' => 1,
'verify' => false,
'http_errors' => false,
'headers' => [
'User-Agent' => 'LaracrawlerBot/1.0 (https://example.com)',
],
],
],
'indexability_audit' => true,
Flags URLs with:
X-Robots-Tag: noindex<meta name="robots" content="noindex">php artisan laracrawler:generate --max-depth=2 --output=public --split --single --no-ping --ping-only --sitemap=sitemap.xml --debug --summary --fresh --queue --validate --audit-indexability
--max-depth → set crawl depth--output → custom output dir--split → force multiple sitemap files--single → force one sitemap.xml--no-ping → skip pinging search engines--ping-only → only ping, no crawl--sitemap → custom sitemap name (with ping-only)--debug → show exclusions in detail--summary → summary of exclusions--fresh → clear cache and recrawl--queue → run crawl in background via jobs--validate → enable link validation--audit-indexability → enable noindex auditsitemap.xml or sitemap-index.xmlsitemap-errors.xml (broken links, invalid alternates, noindex pages)hreflang + x-default)title, caption)title, description)--validate in productionping_targets so Google/Bing auto-refresh fasterpriority_boost in rules for critical pagesindexability_audit to avoid indexing blocked contentThis package is open-sourced software licensed under the MIT license.
