adrians
/
feca-mailshots-plugin
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
feca-mailshots-plugin/requirements/mailshot.md

20 KiB
Raw Blame History

1 Mailshot

Use and comply with requirements/ui_design.md.

1.0 Environment Baseline

  • All writable mailshot tables and mailshot asset binaries are stored in the database named by .env variable MAILSHOTS_REMOTE_MYSQL_DB.
  • This includes mailshots, mailshot_queries, mailshot_attachments, mailshot_last_run, and any mailshot asset tables created for PDF/image assets.
  • In this document, references in the form MAILSHOTS_REMOTE_MYSQL_DB.<table> are normative.
  • Recipient/source data reads used to resolve mailshot recipients and DSL source fields must use .env variable MEMBERS_REMOTE_MYSQL_DB.
  • SMTP/IMAP credentials are sourced from WordPress-side per-user mailshot credential storage.
  • If user credentials are missing, send/run actions must be blocked and the UI must direct users to the WordPress mailshot credentials page.

1.1 Purpose

Provide a Mailshot feature to:

  • manage mailshot definitions
  • preview recipient data and template rendering
  • test mailshots safely
  • execute full mailshot runs

1.2 Data Model and Dependencies

  • Use MAILSHOTS_REMOTE_MYSQL_DB.mailshots as the source of mailshot definitions.
  • Use MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries as the source of recipient/data-source queries.
  • Send mail using per-user SMTP/IMAP credentials from WordPress-side mailshot credential storage.
  • If user credentials are missing, block send/run and direct user to the WordPress mailshot credentials page.
  • Copy successfully sent messages to configured IMAP Sent folder.

1.2.1 Schema Reference: MAILSHOTS_REMOTE_MYSQL_DB.mailshots

Column Type Null Key Notes
id int(11) NO PK Auto-increment
Purpose varchar(100) YES Human-readable mailshot purpose
DataSource varchar(100) NO Data-source name matching mailshot_queries.name
CC varchar(100) YES Optional CC list
BCC varchar(100) YES Optional BCC list
Subject longtext NO Message subject template
Message longtext NO Message body template
PDFAttachment longtext YES Optional PDF attachment html template
AttachmentNames longtext YES JSON array of named entries from MAILSHOTS_REMOTE_MYSQL_DB.mailshot_attachments
PDFFilenameDerivedFrom varchar(128) YES Optional data-source field used as base filename for generated PDFs
ReplyTo varchar(100) YES Optional reply-to address

1.2.2 Schema Reference: MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries

Column Type Null Key Notes
ID int(11) NO PK Auto-increment
name varchar(100) NO UNIQUE Data-source name used by mailshots.DataSource
dsl_text longtext YES DSL sentence (source of truth)
sql longtext NO Legacy column retained; compile target not user-authored

1.3 Navigation and Pages

Under Mailshot, provide child pages:

  • Mailshots
  • Assets
  • Mailshot Recipients Preview
  • Mailshot Test
  • Run Mailshot
  • Download PDF

Under Mailshot > Assets, provide:

  • Mailshot Data Source > Data Sources
  • PDF Assets
  • Attachments

1.3.1 Attachments Data Model

Use MAILSHOTS_REMOTE_MYSQL_DB.mailshot_attachments as reusable uploaded attachments with:

  • id
  • name (unique)
  • file_name
  • mime_type
  • file_bytes
  • created_at
  • updated_at

Persist selected named attachments on each mailshot in MAILSHOTS_REMOTE_MYSQL_DB.mailshots.AttachmentNames (JSON array of attachment names).

1.4 Mailshots Page (MAILSHOTS_REMOTE_MYSQL_DB.mailshots CRUD)

1.4.1 Data Pane

  • Display rows from MAILSHOTS_REMOTE_MYSQL_DB.mailshots.
  • In the list/table view, hide internal id and display user-facing columns (Purpose, DataSource, Subject).
  • Sort the mailshot list by Purpose (ascending).
  • Provide editing for all mutable fields.
  • Highlight unsaved changed fields.
  • Use a separate pane for editing the fields.  Note that the Message field is multiline html-formatted and should be edited using an html edit control.  The design of the page should maximise room for this field
  • Provide a separate html edit control for PDFAttachment (optional template) using the same html editing features as Message.
  • Message and PDF Attachment html editors should open in overlay panes to maximise editing room.
  • Provide PDF Filename Derived From dropdown populated from available data-source fields for the selected DataSource.
  • Persist PDF Filename Derived From as MAILSHOTS_REMOTE_MYSQL_DB.mailshots.PDFFilenameDerivedFrom.
  • The Data Pane also include a table of mailshots,  width limited to the page width,  with truncation of fields.
  • Keep editing responsive; do not trigger per-keystroke save/reload.
  • Data-source pulldowns must be sorted by data-source name (ascending).
  • Show available data-source fields in the editor and allow one-click token insertion into Subject, Message, and PDFAttachment.
  • Token insertion must generate Twig tokens using {{ ... }} syntax (not m_... syntax).
  • Field insert token format default is {{ field_name }} where field_name is a canonical safe field token derived from the selected source field.
  • Subject, Message, and PDFAttachment are always rendered through Twig at render/send time.
  • In editor pane, provide Attachments dropdown (initially blank) sourced from MAILSHOTS_REMOTE_MYSQL_DB.mailshot_attachments.name.
  • Provide Add selected attachment button to append selected item into mailshot attachment list.
  • Allow removing previously selected attachments from the mailshot attachment list.

1.4.2 Action Pane

  • Provide Save button. Enable only when unsaved changes exist.
  • Provide Discard Changes button. Enable only when unsaved changes exist.
  • Provide New button to add a new MAILSHOTS_REMOTE_MYSQL_DB.mailshots row.
  • Provide Delete button. Enable only when one row is selected.
  • On Delete, require confirmation before delete is applied.

1.4.3 Validation

  • Require DataSource to match an existing MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries.name.
  • Prevent save when Subject or Message are blank.
  • Display validation errors in the information pane.

1.5 Template Engine (Twig)

  • Use Twig as the template engine for mailshot rendering.
  • All template tokens and expressions use Twig syntax.
  • Template context is the selected recipient row (plus approved helper functions/filters).
  • Default undefined-variable behavior is non-fatal and renders as empty string while recording a warning.
  • Default escaping policy is:
    • Subject: unescaped text output.
    • Message: unescaped only when explicitly requested in template (| raw), otherwise escaped.
    • PDFAttachment: unescaped only when explicitly requested in template (| raw), otherwise escaped.
  • Control-flow tags ({% if %}, {% for %}, {% set %}) are allowed in Message and PDFAttachment, but disallowed in Subject by default.

Mailshot Data Sources Page

This is defined in requirements/mailshot_data_source.md.

1.6 Mailshot Recipients Preview Page

1.6.1 Selection and Query Execution

  • Provide control to select a data source from MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries.
  • On preview request, execute the selected data-source query and display resulting recipient rows.

1.6.2 Preview Table

  • Display columns from a deterministic field contract derived from all DSL-cited sources, plus any additional source-native columns present in preview output.
  • The deterministic field contract must include all allowed fields for each cited source (built-in or custom).
  • Provide row count and sample-limit indicator in the information pane.
  • Provide generalized recipient sorting controls (Sort by and sort direction) and apply them to the displayed preview rows.
  • If query returns zero rows, display zero-result message (no error state).
  • Provide Download spreadsheet action.
  • On Download spreadsheet, generate recipients.xlsx from rows currently shown in the data pane and download it.
  • In recipients.xlsx, format the header row in bold.
  • In recipients.xlsx, auto-size each column width to fit header/data content.

1.6.3 Safety Rules

  • Preview does not send email.
  • Preview does not persist changes to recipient-source tables.

1.7 Mailshot Test Page

1.7.1 Test Setup

  • Provide control to select a mailshot.
  • Provide control to select one preview/test recipient row from the resolved query output.
  • Provide Test email address input for explicit destination override, defaulting to configured .env value MAILSHOT_TEST_TO_DEFAULT when set.
  • If MAILSHOT_TEST_TO_DEFAULT is unset/blank, initialize Test email address as blank.
  • Rendered Message control must display the html message according to the html formatting.
  • Provide a rendered PDFAttachment html preview pane below the rendered message preview pane.
  • The rendered Message and rendered PDF Attachment preview panes should default to equal heights and support drag-resize via a divider.

1.7.2 Test Mode: Render Only (No Send)

  • Provide action Render Test (No Send).
  • Perform full Twig template rendering for selected row.
  • Display rendered Subject, Message, and PDFAttachment.
  • Do not send SMTP mail.
  • Do not write IMAP Sent copy.

1.7.3 Test Mode: Send Test Email

  • Provide action Send Test Email.
  • Require non-blank Test email address.
  • When Test email address is blank, block send and show validation error in the information pane.
  • Use selected test row only for template rendering context.
  • Send exactly one email to Test email address (not to original recipient address).
  • Copy sent test email to IMAP Sent.
  • Display success/failure status and timestamp.
  • Keep rendered Subject, Message, and PDFAttachment visible after send.

1.8 Run Mailshot Page

1.8.1 Pre-Run Validation

  • Require selected mailshot.
  • Require resolved recipient query.
  • Block run when recipient query returns zero recipients.
  • Display validation failures and do not start run when validation fails.

1.8.2 Execution Semantics

  • On Run Mailshot, execute recipient query and iterate recipients.
  • For each recipient row:
    • perform Twig template rendering
    • send one email to the recipient destination email
    • copy successful sends to IMAP Sent
  • Continue processing remaining recipients after per-recipient failures.

1.8.3 Run Results

  • Display live or final counters:
    • attempted
    • sent
    • failed
    • skipped (if applicable)
  • Provide downloadable or copyable run log in the UI.
  • Include per-recipient failure reason where available.

1.8.4 Last Run Retry Table

  • Persist the outcome of the most recent run only in MAILSHOTS_REMOTE_MYSQL_DB.mailshot_last_run.
  • Clear MAILSHOTS_REMOTE_MYSQL_DB.mailshot_last_run when starting a new Run Mailshot execution.
  • Each row in MAILSHOTS_REMOTE_MYSQL_DB.mailshot_last_run must store:
    • mailshot_id
    • recipient_key (stable non-email identifier)
    • recipient_key_field (source field used to derive key)
    • recipient_email_last (most recent resolved destination email)
    • status
    • error_message and/or warning_message
    • attempt_count
    • last_attempt_at
  • Mailshots page must display the latest rows for the selected mailshot from MAILSHOTS_REMOTE_MYSQL_DB.mailshot_last_run.
  • Mailshots page must provide:
    • Retry Failed Sends button to retry all failed rows from the selected mailshot.
    • per-row Retry button to retry one failed row by recipient key.
  • Retry must resolve recipient data from the current data source and current template values at retry time (not snapshot payload from original run).
  • Email address must not be used as the retry key.

1.9 Download PDF Page

1.9.1 Setup and Validation

  • Provide control to select a mailshot.
  • Require selected mailshot.
  • Require resolved recipient query.
  • Block generation when recipient query returns zero recipients.
  • Block generation when selected mailshot has blank PDFAttachment template.

1.9.2 Output Options

  • Provide action to generate and download individual recipient PDFs as a single .zip.
  • Provide action to generate and download a single merged PDF containing all recipient PDFs.
  • Use html-to-pdf conversion of rendered PDFAttachment content per recipient row.

1.9.3 Results and Logging

  • Display generation success/failure status in the UI.
  • Generate downloadable files directly (no per-row log table required on this page).

1.10 Mailshot Queries Page

Provide a CRUD page for the MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries table.

1.10.1 DSL Built-in Sources

Built-in DSL sources include:

  • contacts
  • accounts
  • renewals

renewals is a first-class source and must be joinable to:

  • accounts via renewals.account_id = accounts.ID
  • contacts through accounts (renewals.account_id = contacts.Accountid)

1.10.2 DSL Renewal and Membership Filters

DSL filters include:

  • selected-renewal
  • pending-renewal
  • fen1-contact
  • fen2-contact
  • member-or-affiliate-or-parish-council

selected-renewal uses saved field values (renewals.selected = true). pending-renewal filters rows to renewals.status = 'pending'.

2.0 No-Effect and Error Handling

  • If Save is clicked with no changes, no effect.
  • If Discard Changes is clicked with no changes, no effect.
  • If a selected mailshot references a missing MAILSHOTS_REMOTE_MYSQL_DB.mailshot_queries row, block preview/test/run and display error.
  • If SMTP send fails for one recipient, record error and continue with next recipient.
  • If IMAP copy fails after successful SMTP send, mark warning for that recipient.
  • For any DSL sentence, all fields from all cited sources must be available for token insertion and template rendering; missing values may render empty, but fields must not disappear from mailshot field availability.
  • Twig template syntax errors must not crash the page or run; show clear error identifying mailshot and template field (Subject, Message, or PDFAttachment).
  • Default behavior on template syntax error is fail-fast for that recipient (skip send for recipient, continue run for others) and record error.
  • Retry actions must update MAILSHOTS_REMOTE_MYSQL_DB.mailshot_last_run rows in-place (incrementing attempt_count and updating status/messages/timestamp).

2.0.1 IMAP Sent-Copy Contract

  • Use per-user configured IMAP sent-folder path from WordPress mailshot credentials.
  • IMAP folder names must be encoded/decoded using IMAP UTF-7 rules.
  • For each successful SMTP send, attempt a single IMAP append for sent-copy persistence.
  • IMAP append failure must not change SMTP send outcome; classify as warning, not failed.
  • IMAP append retries are not automatic in run loop; retry occurs only via explicit operator retry action.
  • Sent-copy operations must be idempotent per attempt id; duplicate append for the same attempt id is not allowed.

2.1 PDF Assets

2.1.1 Purpose

Provide reusable image assets that can be inserted into PDFAttachment html templates using Twig syntax and resolved at PDF render time.

2.1.2 Navigation and Page

Under Mailshot, provide page PDF Assets.

2.1.3 Data Model

Provide CRUD storage for PDF assets with at least:

  • id (primary key)
  • name (unique asset identifier used in token marker)
  • file_name (original upload filename)
  • mime_type
  • file_bytes (or storage reference)
  • width_mm
  • height_mm
  • justification (left, right, in-place)
  • created_at
  • updated_at

2.1.4 PDF Assets Page Behavior

  • List assets in a table.
  • Support Create, Read, Update, Delete.
  • Upload graphic file during create/update.
  • Edit dimensions in millimetres.
  • Edit justification (left, right, in-place).
  • Validate file type and size constraints.

2.1.5 Mailshots PDF Editor Integration

  • In the PDF Attachment editor overlay, add an insert-asset action with dropdown of available assets.
  • On insert, place Twig token syntax at cursor position.
  • Default insert format is {{ pdf_asset('asset_name') }}.
  • Asset names used in inserted tokens must be normalized to safe token characters.

2.1.6 Rendering Behavior

  • On rendering PDFAttachment html (Mailshot Test preview/send, Run Mailshot, Download PDF), resolve each inserted PDF asset Twig token to rendered image html for that asset.
  • Render image using configured dimensions (width_mm, height_mm) and configured justification.
  • If template references missing asset, do not fail entire run; record warning and render explicit missing-asset placeholder html.
  • Missing asset placeholder default is [missing pdf asset: <asset_name>].

2.1.7 PDF Filename Derivation

  • For each rendered recipient pdf (download and email attachment), if PDFFilenameDerivedFrom is set, use row value from that field as base filename.
  • Sanitize filename to safe filename characters.
  • If PDFFilenameDerivedFrom is blank or field value resolves blank, use fallback naming strategy.
  • Ensure filename uniqueness within a generated batch (append numeric suffix as needed).

2.2 PDF Assets Decisions

  • Asset binaries are stored in DB table (LONGBLOB) for portability and backup simplicity.
  • Allowed upload formats: png, jpg/jpeg, svg.
  • Scope/ownership: global asset library, all authenticated users may CRUD.
  • Asset naming: unique case-insensitive name; renaming does not auto-migrate existing asset insertions in templates and should warn user.
  • Marker syntax is removed. PDF asset insertion uses Twig syntax.
  • Helper name is pdf_asset(name) as a global Twig helper, not a filter.
  • Dimension/aspect behavior: fixed width+height in mm with object-fit: contain (no crop/stretch distortion).
  • Justification semantics: in-place renders inline at marker location; left and right render block-aligned elements on their own line (no text-wrap behavior).
  • Upload limits: max file size 2MB, max image dimensions 4000x4000.
  • Missing asset token behavior: continue processing, emit warning, render explicit missing-asset placeholder.
  • Delete behavior for in-use assets: block delete when asset helper usage is referenced by any PDFAttachment template and show referencing mailshot count.

2.3 Mailshot Attachments

2.3.1 Page and Layout

Under Mailshot > Assets, provide page Attachments.

  • Data pane: table of attachments sorted by name.
  • CRUD pane (right): edit name and uploaded file.
  • Provide actions: Save, New, Delete.

2.3.2 CRUD and Validation

  • name is required and unique.
  • Create requires uploaded file bytes.
  • Update allows metadata-only edits without replacing file.
  • Delete must be blocked while referenced by any MAILSHOTS_REMOTE_MYSQL_DB.mailshots.AttachmentNames.

2.3.3 Send/Run Behavior

  • On Mailshot Test > Send Test Email, include all selected named attachments.
  • On Run Mailshot, include all selected named attachments for each recipient email.
  • If a selected attachment name is missing, continue send/run and record warning.