NNTmux

NNTmux is a modern Usenet indexer built on Laravel, designed for high performance and scalability. It automatically scans Usenet, collects headers, and organizes them into searchable releases. The project is actively maintained and features multi-threaded processing, advanced search, and a web-based front-end with API access.

This project is a fork of newznab plus and nZEDb, with significant improvements:

• Multi-threaded processing (header retrieval, release creation, post-processing)
• Advanced search (name, subject, category, post-date)
• Intelligent local caching of metadata (TMDB, TVDB, TVMaze, Trakt, IMDB)
• Tmux engine for thread, database, and performance monitoring
• Image and video sample support
• Modern frontend stack: Vite, Tailwind CSS, Alpine.js, Livewire
• Full-text search via Elasticsearch or ManticoreSearch
• Dockerized development via Laravel Sail
• RESTful API compatible with newznab standard

Table of Contents

• Prerequisites
• Installation
• Configuration
• Database Setup
• Search Engines
• Console Commands
• IRC Pre Channels
• TV & Movie Processing
• API
• Docker & Development
• Troubleshooting
• Contributing
• License

Prerequisites

• System administration experience (Linux recommended)
• PHP 8.4+ with extensions: curl, json, pdo_mysql, openssl, mbstring, xml, zip, gd, intl, pcntl
• MariaDB 10.6+ or MySQL 8+ (PostgreSQL not supported)
• Composer 2.x
• Node.js 18+ and npm for frontend assets
• nginx or Apache web server
• Optional: tmux (for multi-threaded processing)
• Optional: unrar, 7zip, ffmpeg, mediainfo (for post-processing)

Recommended Hardware

| Scale | RAM | CPU | Disk | |-------|-----|-----|------| | Small (<1M releases) | 16GB | 4 cores | 100GB SSD | | Medium (1-10M releases) | 32GB | 8 cores | 250GB SSD | | Large (10M+ releases) | 64GB+ | 16+ cores | 500GB+ NVMe |

Installation

Quick Start

1. Clone the repository:

   git clone https://github.com/NNTmux/newznab-tmux.git
   cd newznab-tmux
   

2. Install PHP dependencies:

   composer install --no-dev --optimize-autoloader
   

3. Copy and configure environment:

   cp .env.example .env
   php artisan key:generate
   

4. Configure your .env file (see Configuration)

5. Run database migrations:

   php artisan migrate
   

6. Install frontend assets:

   npm install
   npm run build
   

7. Set permissions:

   chmod -R 775 storage bootstrap/cache
   chown -R www-data:www-data storage bootstrap/cache
   

For detailed installation instructions, see the Ubuntu Install Guide.

Configuration

Essential .env Settings

# Application APP_NAME=NNTmux APP_ENV=production APP_DEBUG=false APP_URL=https://your-domain.com # Database DB_CONNECTION=mysql DB_HOST=127.0.0.1 DB_PORT=3306 DB_DATABASE=nntmux DB_USERNAME=nntmux DB_PASSWORD=your_secure_password # Usenet Server NNTP_SERVER=news.your-provider.com NNTP_PORT=563 NNTP_SSLENABLED=true NNTP_USERNAME=your_username NNTP_PASSWORD=your_password # API Keys (obtain from respective services) TMDB_API_KEY=your_tmdb_key TVDB_API_KEY=your_tvdb_key TVMAZE_API_KEY= # Optional, no key required for basic usage TRAKT_CLIENT_ID=your_trakt_client_id TRAKT_CLIENT_SECRET=your_trakt_secret OMDB_API_KEY=your_omdb_key FANART_API_KEY=your_fanart_key GIANTBOMB_API_KEY=your_giantbomb_key # Search Engine (choose one) SEARCH_ENGINE=manticore # Options: manticore, elasticsearch MANTICORE_HOST=127.0.0.1 MANTICORE_PORT=9308 # Or for Elasticsearch: # ELASTICSEARCH_HOST=127.0.0.1 # ELASTICSEARCH_PORT=9200 # IRC Pre Scraping (optional) SCRAPE_IRC_SERVER=irc.synirc.net SCRAPE_IRC_PORT=6697 SCRAPE_IRC_TLS=true SCRAPE_IRC_USERNAME=YourUniqueNick

Queue Configuration

NNTmux uses Laravel's queue system for background processing:

QUEUE_CONNECTION=redis REDIS_HOST=127.0.0.1 REDIS_PORT=6379

For high-volume processing, consider using Laravel Horizon.

Database Setup

Initial Setup

# Run migrations
php artisan migrate

# Seed initial data (categories, groups, etc.)
php artisan db:seed

Database Tuning

For large-scale indexing, proper database tuning is critical. Key settings:

# /etc/mysql/mariadb.conf.d/99-nntmux.cnf
[mysqld]
innodb_buffer_pool_size = 8G          # 50-70% of available RAM
innodb_log_file_size = 1G
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
query_cache_type = 0
query_cache_size = 0
max_connections = 500
tmp_table_size = 256M
max_heap_table_size = 256M

Use mysqltuner.pl for recommendations:

wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl
perl mysqltuner.pl

Collation Migration

For proper Unicode support (emojis, special characters):

php artisan nntmux:convert-collation utf8mb4_unicode_ci

Search Engines

NNTmux supports two full-text search engines:

ManticoreSearch (Recommended)

# Install ManticoreSearch
wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
sudo dpkg -i manticore-repo.noarch.deb
sudo apt update
sudo apt install manticore

# Configure in .env
SEARCH_ENGINE=manticore
MANTICORE_HOST=127.0.0.1
MANTICORE_PORT=9308

# Build indexes
php artisan nntmux:index-manticore

Elasticsearch

# Install Elasticsearch
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update && sudo apt install elasticsearch

# Configure in .env
SEARCH_ENGINE=elasticsearch
ELASTICSEARCH_HOST=127.0.0.1
ELASTICSEARCH_PORT=9200

# Build indexes
php artisan nntmux:index-elasticsearch

Console Commands

NNTmux provides numerous Artisan commands for management and maintenance.

Release Processing

# Start the tmux processing engine
php artisan tmux:start

# Stop tmux processing
php artisan tmux:stop

# Process releases manually
php artisan nntmux:process-releases

# Update release names
php artisan nntmux:update-releases

TV & Movie Processing

# Reprocess unmatched TV releases
php artisan nntmux:reprocess-tv

# Refresh TV episodes for a specific show
php artisan tv:refresh-episodes --video-id=12345

# Refresh all shows with missing seasons
php artisan tv:refresh-episodes --missing-seasons

# Search by title and refresh
php artisan tv:refresh-episodes --title="Show Name"

# Delete existing and re-fetch from specific provider
php artisan tv:refresh-episodes --video-id=12345 --provider=tmdb --delete-existing

# Dry run to preview changes
php artisan tv:refresh-episodes --missing-seasons --dry-run

Database Maintenance

# Optimize tables
php artisan nntmux:optimize-tables

# Clean old releases
php artisan nntmux:cleanup --days=365

# Purge Laravel Pulse data (if using Pulse)
php artisan pulse:purge

# Clear application caches
php artisan cache:clear
php artisan config:clear
php artisan view:clear
php artisan route:clear

User Management

# Create admin user
php artisan nntmux:create-admin

# Reset user password
php artisan nntmux:reset-password [email protected]

IRC Pre Channels

NNTmux can scrape IRC pre channels for early release information.

Active IRC Networks

| Network | Server | Ports | SSL | |---------|--------|-------|-----| | SynIRC | irc.synirc.net | 6667, 6697, 7001 | Yes | | p2p-net | irc.p2p-net.eu | 6697, 7000 | Yes | | lillesky | irc.lillesky.org | 6667, 7000 | Yes | | Abjects | irc.abjects.net | 6667, 6697 | Yes |

Configuration

SCRAPE_IRC_SERVER=irc.synirc.net SCRAPE_IRC_PORT=6697 SCRAPE_IRC_TLS=true SCRAPE_IRC_USERNAME=YourUniqueNickname SCRAPE_IRC_REALNAME=Your Name SCRAPE_IRC_CHANNELS=#PreNNTmux,#nZEDbPRE

PreDB API Alternatives

Instead of or in addition to IRC, you can use PreDB APIs:

• predb.ovh - Public API with RSS feeds: https://predb.ovh/api/v1/
• predb.net - Comprehensive API: https://predb.net/api-documentation/
• predb.me - Public PreDB with search
• predb.live - NFO database included

TV & Movie Processing

NNTmux fetches metadata from multiple sources with fallback support.

Provider Priority

1. TMDB (The Movie Database) - Primary for movies and TV
2. TVDB (TheTVDB) - Fallback for TV shows
3. TVMaze - Additional TV metadata
4. Trakt - User ratings and additional data
5. OMDB/IMDB - Movie ratings and legacy data

Episode Matching

The TV processing pipeline automatically:

• Matches releases to shows by name
• Downloads episode information for matched shows
• Detects and fetches missing seasons when processing new releases

If episodes aren't matching properly:

# Check what video ID a show has
php artisan tinker
>>> DB::table('videos')->where('title', 'like', '%Show Name%')->get(['id', 'title', 'tmdb']);

# Refresh episodes for that show
php artisan tv:refresh-episodes --video-id=<ID>

API

NNTmux provides a newznab-compatible API for integration with download clients and media managers (Sonarr, Radarr, etc.).

Endpoints

GET /api?t=caps           # Server capabilities
GET /api?t=search&q=      # Search releases
GET /api?t=tvsearch       # TV search
GET /api?t=movie          # Movie search
GET /api?t=music          # Music search
GET /api?t=book           # Book search
GET /api?t=details&id=    # Release details
GET /api?t=getnzb&id=     # Download NZB

API Keys

Users obtain API keys from their profile page. Configure per-user rate limits in the admin panel.

For detailed API documentation, see the NNTmux API v2 Wiki.

Docker & Development

Laravel Sail (Docker)

# Start containers
./vendor/bin/sail up -d

# Run artisan commands
./vendor/bin/sail artisan migrate

# Stop containers
./vendor/bin/sail down

Frontend Development

# Install dependencies
npm install

# Development with hot reload
npm run dev

# Production build
npm run build

Code Style

# PHP formatting (Laravel Pint)
./vendor/bin/pint

# JavaScript/Vue linting
npm run lint

Agent Skills

The repository keeps the shared agent skill baseline in skills-lock.json. The generated Linux installer installs those skills globally under ~/.agents/skills and bridges them into Codex under ~/.codex/skills.

Prerequisites: Bash, Node.js/npm (npx), and network access to the skill repositories.

# Regenerate the installer after changing skills-lock.json
./scripts/generate-global-skills-installer.sh

# Preview global installs and Codex bridge changes
./scripts/install-global-skills.sh --dry-run

# Install globally and create Codex symlinks
./scripts/install-global-skills.sh

# Copy skills into Codex instead of creating symlinks
./scripts/install-global-skills.sh --copy

Set SKILLS_GLOBAL_DIR or CODEX_HOME when using non-default global directories. The installer never removes or overwrites unmanaged existing skill paths; resolve reported conflicts manually and rerun it.

Troubleshooting

Common Issues

Releases not being created:

# Check for errors in logs
tail -f storage/logs/laravel.log

# Verify NNTP connection
php artisan nntmux:test-nntp

# Check group status
php artisan tinker
>>> DB::table('usenet_groups')->where('active', 1)->count();

TV/Movie not matching:

# Reprocess specific release
php artisan nntmux:reprocess-release --id=12345

# Check API keys
php artisan nntmux:test-apis

High disk usage from Laravel Pulse:

# Check Pulse table sizes
mysql -e "SELECT table_name, ROUND(data_length/1024/1024, 2) as 'Size (MB)' FROM information_schema.tables WHERE table_schema='nntmux' AND table_name LIKE 'pulse%';"

# Purge old data
php artisan pulse:purge

Permission issues:

# Fix ownership
sudo chown -R www-data:www-data storage bootstrap/cache

# Fix permissions
sudo chmod -R 775 storage bootstrap/cache

Logs

• Application logs: storage/logs/laravel.log
• Tmux logs: misc/update/tmux/logs/
• nginx logs: /var/log/nginx/
• PHP-FPM logs: /var/log/php-fpm/

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development Setup

1. Fork the repository
2. Create a feature branch: git checkout -b feature/your-feature
3. Make your changes
4. Run tests: php artisan test
5. Submit a pull request

Support

• Discord: Join our server
• GitHub Issues: Report bugs
• Wiki: Documentation

License

NNTmux is open-source software licensed under the GPL v3. External libraries include their own licenses in their respective folders.