Heratio Help Center article. Category: Plugin Reference.

ahgIiifPlugin - Technical Documentation

Overview

The ahgIiifPlugin provides comprehensive IIIF (International Image Interoperability Framework) capabilities for Heratio, including manifest generation, deep zoom viewing, media streaming with on-the-fly transcoding, annotation support, collection management, and authentication (IIIF Auth API 1.0). The plugin supports images, PDFs, multi-page TIFFs, 3D models, and audio/video content.

Version: 1.3.0 Category: Media/Viewing Dependencies: atom >= 2.8.0, PHP >= 8.1, atom-framework

Optional Dependencies:

• Cantaloupe (IIIF Image Server for deep zoom tiling)
• FFmpeg (media transcoding and metadata extraction)
• Whisper (audio/video transcription)
• ImageMagick (PSD/RAW conversion)
• LibreOffice (Office document → PDF conversion)

Architecture

Component Diagram

┌─────────────────────────────────────────────────────────────────────────────────┐
│                           ahgIiifPlugin                                          │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  ┌────────────────────────────────────────────────────────────────────────────┐ │
│  │                       PRESENTATION LAYER                                   │ │
│  │  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │ │
│  │  │ IiifViewer   │ │ Collection   │ │ Auth Admin   │ │ Viewer Settings  │ │ │
│  │  │ Manager (JS) │ │ Templates    │ │ Templates    │ │ Templates        │ │ │
│  │  │ (ES6 module) │ │              │ │              │ │                  │ │ │
│  │  └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────┘ │ │
│  └────────────────────────────────────────────────────────────────────────────┘ │
│                                      │                                           │
│                                      ▼                                           │
│  ┌────────────────────────────────────────────────────────────────────────────┐ │
│  │                      CONTROLLER LAYER                                      │ │
│  │  ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ │
│  │  │ iiif/actions   │ │ iiifCollection│ │ iiifAuth      │ │ media/actions  │ │ │
│  │  │               │ │ /actions      │ │ /actions      │ │               │ │ │
│  │  │ manifest()    │ │ index()       │ │ login()       │ │ stream()      │ │ │
│  │  │ manifestById()│ │ create()      │ │ token()       │ │ download()    │ │ │
│  │  │ annotations() │ │ edit()        │ │ logout()      │ │ transcribe()  │ │ │
│  │  │ annotationsModify()│ │ manifest()│ │ confirm()     │ │ convert()     │ │ │
│  │  │ compare()     │ │ addItems()    │ │ check()       │ │ extract()     │ │ │
│  │  │ settings()    │ │ reorder()     │ │ protect()     │ │ snippets()    │ │ │
│  │  └───────────────┘ └───────────────┘ └───────────────┘ └───────────────┘ │ │
│  └────────────────────────────────────────────────────────────────────────────┘ │
│                                      │                                           │
│                                      ▼                                           │
│  ┌────────────────────────────────────────────────────────────────────────────┐ │
│  │                       SERVICE LAYER                                        │ │
│  │  ┌──────────────────────┐ ┌──────────────────────┐ ┌────────────────────┐ │ │
│  │  │ IiifAnnotationService│ │ IiifCollectionService │ │ IiifAuthService    │ │ │
│  │  │ (CRUD + W3C format)  │ │ (CRUD + hierarchy)   │ │ (tokens + access)  │ │ │
│  │  └──────────────────────┘ └──────────────────────┘ └────────────────────┘ │ │
│  │  ┌──────────────────────┐ ┌──────────────────────┐ ┌────────────────────┐ │ │
│  │  │ TranscriptionService │ │ MediaConversionService│ │IiifManifestV3Svc  │ │ │
│  │  │ (Whisper integration)│ │ (FFmpeg/IM/LO)       │ │(v3 manifest gen)  │ │ │
│  │  └──────────────────────┘ └──────────────────────┘ └────────────────────┘ │ │
│  └────────────────────────────────────────────────────────────────────────────┘ │
│                                      │                                           │
│                                      ▼                                           │
│  ┌────────────────────────────────────────────────────────────────────────────┐ │
│  │                       HELPER LAYER                                         │ │
│  │                    IiifViewerHelper.php                                     │ │
│  │  ┌───────────────────────────────────────────────────────────────────────┐ │ │
│  │  │ render_iiif_viewer()  │ get_iiif_base_url() │ get_preferred_viewer() │ │ │
│  │  │ render_3d_viewer()    │ get_cantaloupe_url() │ has_3d_models()       │ │ │
│  │  │ render_av_viewer()    │ get_digital_obj_url() │ render_controls()    │ │ │
│  │  │ render_pdf_viewer()   │ render_thumbnail_strip() │ render_toggle()  │ │ │
│  │  └───────────────────────────────────────────────────────────────────────┘ │ │
│  └────────────────────────────────────────────────────────────────────────────┘ │
│                                      │                                           │
│                                      ▼                                           │
│  ┌────────────────────────────────────────────────────────────────────────────┐ │
│  │                       DATA LAYER                                           │ │
│  │         Illuminate\Database\Capsule\Manager (Laravel Query Builder)        │ │
│  └────────────────────────────────────────────────────────────────────────────┘ │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘

Database Schema

Tables

Entity Relationships

information_object (Heratio core)
    │
    ├──► iiif_collection_item.object_id
    ├──► iiif_annotation.object_id
    ├──► iiif_auth_resource.object_id
    └──► iiif_auth_access_log.object_id

iiif_collection ──► iiif_collection_i18n (1:N)
                ──► iiif_collection_item (1:N, CASCADE)
                ──► iiif_collection (self-ref parent_id)

iiif_annotation ──► iiif_annotation_body (1:N, CASCADE)

iiif_auth_service ──► iiif_auth_token (1:N)
                  ──► iiif_auth_resource (1:N)

iiif_ocr_text ──► iiif_ocr_block (1:N)

Routing

Routes are registered in ahgIiifPluginConfiguration.class.php via the routing.load_configuration event.

Manifest Routes

Annotation Routes

Collection Routes

Media Routes

Auth Routes (IIIF Auth API 1.0)

Manifest Generation

Process

1. Query information_object by slug or ID (Laravel Query Builder)
2. Get all digital_object records for the object
3. Build IIIF identifier: str_replace('/', '_SL_', $path . $name)
4. Check for multi-page TIFF (probe Cantaloupe for pages 2-100)
5. Generate canvases (one per digital object, or one per page for multi-page TIFFs)
6. Return IIIF Presentation API 3.0 manifest (default), or 2.1 with ?format=2

Multi-Page TIFF Detection

// Probe Cantaloupe for page 2
$page2InfoUrl = "{$cantaloupeBaseUrl}/iiif/2/{$cantaloupeId};2/info.json";
$page2Info = @file_get_contents($page2InfoUrl);
if ($page2Info !== false) {
    $isMultiPageTiff = true;
    // Continue probing up to page 100
}

Manifest Structure (IIIF 2.1)

{
  "@context": "http://iiif.io/api/presentation/2/context.json",
  "@type": "sc:Manifest",
  "@id": "https://host/iiif/manifest/slug",
  "label": "Object Title",
  "sequences": [{
    "@type": "sc:Sequence",
    "canvases": [{
      "@type": "sc:Canvas",
      "label": "Page 1",
      "width": 4000,
      "height": 3000,
      "images": [{
        "@type": "oa:Annotation",
        "resource": {
          "@type": "dctypes:Image",
          "service": {
            "@context": "http://iiif.io/api/image/2/context.json",
            "@id": "https://host/iiif/2/identifier",
            "profile": "http://iiif.io/api/image/2/level2.json"
          }
        }
      }]
    }]
  }]
}

Manifest Structure (IIIF 3.0 — Default)

{
  "@context": "http://iiif.io/api/presentation/3/context.json",
  "id": "https://host/iiif/v3/manifest/slug",
  "type": "Manifest",
  "label": {"en": ["Object Title"]},
  "summary": {"en": ["Scope and content text"]},
  "metadata": [...],
  "rights": "https://creativecommons.org/licenses/by/4.0/",
  "requiredStatement": {
    "label": {"none": ["Attribution"]},
    "value": {"en": ["Institution Name"]}
  },
  "provider": [{
    "id": "https://host/repository-slug",
    "type": "Agent",
    "label": {"en": ["Institution Name"]}
  }],
  "seeAlso": [{
    "id": "https://host/record-slug",
    "type": "Dataset",
    "format": "text/html"
  }],
  "items": [{
    "id": "https://host/.../canvas/1",
    "type": "Canvas",
    "width": 4000,
    "height": 3000,
    "items": [{
      "type": "AnnotationPage",
      "items": [{
        "type": "Annotation",
        "motivation": "painting",
        "body": {
          "type": "Image",
          "service": [
            {"type": "ImageService2", "profile": "level2"},
            {"type": "ImageService3", "profile": "level2"}
          ]
        }
      }]
    }]
  }]
}

Multi-language labels are automatically populated from all available information_object_i18n cultures.

Viewer Manager (JavaScript)

File: web/js/iiif-viewer-manager.js

ES6 module that manages all viewer types with lazy-loading and preference persistence.

Supported Viewers

Viewer Selection Logic

Content type detected
    │
    ├─ PDF → PDF.js (override user preference)
    ├─ Audio/Video → HTML5 player (override user preference)
    ├─ 3D model → model-viewer (override user preference)
    └─ Image → User preference (OpenSeadragon or Mirador)
                Saved in localStorage

Key Methods

class IiifViewerManager {
    constructor(containerId, options)
    initOpenSeadragon(manifestUrl)
    initMirador(manifestUrl)
    initPdfViewer(pdfUrl)
    init3DViewer(modelUrl)
    switchViewer(viewerType)
    toggleFullscreen()
    toggleAnnotations()
    showError(message)
    destroy()
}

Media Streaming

Transcoding (FFmpeg)

When a file's MIME type is not browser-native, FFmpeg transcodes in real-time:

Video → MP4:

ffmpeg -y -i INPUT \
  -c:v libx264 -preset ultrafast -tune zerolatency -crf 23 \
  -c:a aac -b:a 128k \
  -movflags frag_keyframe+empty_moov+faststart \
  -f mp4 pipe:1

Audio → MP3:

ffmpeg -y -i INPUT \
  -c:a libmp3lame -b:a 192k \
  -f mp3 pipe:1

Formats Requiring Transcoding

Range Request Support

For native formats, HTTP Range requests enable seeking:

if (isset($_SERVER['HTTP_RANGE'])) {
    preg_match('/bytes=(\d*)-(\d*)/', $_SERVER['HTTP_RANGE'], $matches);
    http_response_code(206);
    header("Content-Range: bytes {$start}-{$end}/{$fileSize}");
}

Format Conversion (non-streaming)

Conversions cached in /uploads/conversions/.

Transcription (Whisper)

Background Job Pattern

1. Create lock file to prevent duplicate jobs
2. Generate temporary PHP script in /tmp
3. Launch as background process: php script.php >> log 2>&1 &
4. Client polls /media/transcription/:id every 5 seconds
5. Status file written on completion

TranscriptionService

• Python/Whisper integration
• Models: tiny, base, small, medium, large
• Language auto-detection or manually specified
• Output: segments (timestamped), full_text, confidence

Export Formats

Annotation Service

IiifAnnotationService Methods

getAnnotationsForObject(int $objectId): array
getAnnotationsForCanvas(string $canvasId): array
createAnnotation(array $data): int
updateAnnotation(int $annotationId, array $data): bool
deleteAnnotation(int $annotationId): bool
formatAsAnnotationPage(array $annotations, int $objectId): array
formatAnnotationAsIiif(object $annotation): array
parseAnnotoriousAnnotation(array $annoData, int $objectId): array
syncResearchAnnotation(int $researchAnnotationId, ?int $iiifAnnotationId = null): int

Motivations

Selectors (JSON stored in target_selector)

• FragmentSelector: xywh=x,y,w,h (rectangle)
• SvgSelector: <svg>...</svg> (polygon, freehand)

IIIF Auth API 1.0

Auth Profiles

Authentication Flow

Client                                              Server
  │                                                    │
  │  1. Request manifest                               │
  │───────────────────────────────────────────────────►│
  │                                                    │
  │  2. Manifest with service block                    │
  │◄───────────────────────────────────────────────────│
  │                                                    │
  │  3. Open popup: /iiif/auth/login/:service          │
  │───────────────────────────────────────────────────►│
  │                                                    │
  │  4. Login page / Clickthrough terms                │
  │◄───────────────────────────────────────────────────│
  │                                                    │
  │  5. User authenticates/agrees                      │
  │───────────────────────────────────────────────────►│
  │                                                    │
  │  6. Set cookie, close popup                        │
  │◄───────────────────────────────────────────────────│
  │                                                    │
  │  7. Token request: /iiif/auth/token/:service       │
  │───────────────────────────────────────────────────►│
  │                                                    │
  │  8. Access token response                          │
  │◄───────────────────────────────────────────────────│
  │                                                    │
  │  9. Request image with token                       │
  │───────────────────────────────────────────────────►│
  │                                                    │
  │  10. Full resolution image                         │
  │◄───────────────────────────────────────────────────│

IiifAuthService Methods

checkAccess(int $objectId, ?int $userId = null): array
// Returns: ['allowed' => bool, 'degraded' => bool, 'service' => ?array]

requestToken(string $serviceName, ?int $userId, ?string $messageId): array
validateCurrentToken(?int $serviceId = null): ?object
logout(): bool
cleanupExpiredTokens(): int

setObjectAuth(int $objectId, string $serviceName, array $options = []): bool
// Options: apply_to_children, degraded_access, degraded_width, notes

removeObjectAuth(int $objectId, ?string $serviceName = null): bool

Token Security

• Tokens hashed with SHA-256 before storage
• Cookies: HttpOnly, Secure, SameSite=None
• Configurable TTL per service (default: 1 hour)
• Automatic cleanup of expired tokens

Access Inheritance

Repository Level
    └── Fonds (apply_to_children=true)
        └── Series (inherits)
            └── File (inherits)
                └── Item (inherits)

Nginx Integration

extensions.conf

# Media streaming - buffering off for real-time transcoding
location ~ ^/media/stream/ {
    fastcgi_buffering off;
    fastcgi_read_timeout 600;
    # → PHP-FPM
}

# Format conversion - 120s timeout for LibreOffice/ImageMagick
location ~ ^/media/convert/ {
    fastcgi_read_timeout 120;
    # → PHP-FPM
}

# Transcription - 600s timeout for Whisper
location ~ ^/media/transcribe/ {
    fastcgi_read_timeout 600;
    # → PHP-FPM
}

# Static viewer assets - 7 day cache
location /atom-framework/src/Extensions/IiifViewer/public/ {
    expires 7d;
}

# Optional: Cantaloupe IIIF Image API proxy
# location /iiif/ {
#     proxy_pass http://127.0.0.1:8182/iiif/;
# }

Helper Functions (IiifViewerHelper.php)

Main Entry Point

render_iiif_viewer($resource, $options = [])

Detects content type, selects viewer, builds HTML with container + controls + JavaScript initialization. Returns complete HTML string.

Content-Specific Renderers

UI Components

URL Helpers

Plugin Integration Points

All integration is guarded with try/catch or class_exists() — the plugin degrades gracefully when optional plugins are missing.

CSP Compliance

All inline <script> and <style> tags use the CSP nonce:

$n = sfConfig::get('csp_nonce', '');
$nonceAttr = $n ? preg_replace('/^nonce=/', ' nonce="', $n) . '"' : '';
echo '<script type="module"' . $nonceAttr . '>';

All viewer libraries are bundled locally — no CDN whitelisting required for IIIF functionality. External CDN domains are only needed if other plugins load external scripts.

Configuration (sfConfig)

Viewer Settings (iiif_viewer_settings table)

Static Assets

Bundled Libraries

Bundled Libraries

All viewer libraries are served locally from web/js/vendor/ — no CDN dependencies at runtime:

Error Handling

Troubleshooting

Version History

Part of the Heratio AHG Framework