Discover RSS 2.0, RSS 1.0\/RDF, Atom, JSON Feed 1\/1.1 and iCalendar feeds and automatically import articles and media.<\/p>\n
Administrator accounts cannot run imports. Select an enabled non-administrator account\nwith only the required import permissions. This is checked when saving settings and\nbefore polling, including for existing feeds whose account is later promoted to admin.<\/p>\n
Enable Import feeds<\/strong> in Modules and open Content > Import feeds<\/strong>. The module\nrequires Managing feeds requires Settings are stored in the feed table. There are no Imports create published resources and media placeholder resources. Inline images\nbecome Each task rechecks the feed's current import account and fetch credentials, resource\npermissions, pending URL and existing medium. Deleted resources and locally replaced\nmedia are not overwritten. Download failures retry up to sixteen times after the initial attempt. Delays\ndouble from one minute up to a fixed twelve-hour cap: 1, 2, 4, 8, 16, 32, 64,\n128, 256 and 512 minutes, followed by six twelve-hour waits (89 hours 3 minutes\ntotal, excluding request execution and scheduling delays). A longer Native Zotonic resource feeds already use the core importer's persistent media tasks.<\/p>\n For each inline image, the importer selects one variant, preferring the largest\n Discovery can use the current user's registered OAuth2 consumer credentials.\nSaved feeds use their selected import user's credentials for previews and polling.\nTokens are resolved per destination host and port, only over HTTPS, and are never\nstored in feed settings. Missing credentials leave a request anonymous. Native\nimports save their fetch preference for core reimports and medium downloads.<\/p>\n Development sites allow local destinations and self-signed TLS certificates.\nTransport, redirects and certificate checks use the shared Polling stores the source's Atom and RSS Later polls process each complete page and stop following links when that page\ncontains a previously successfully imported entry. Deleted tracked entries also\ncount as seen. This fetches new published entries, but is not a history or change\nsynchronization mechanism: older edits and backdated articles may be missed.<\/p>\n Continuation requests use the same URL, credential and response-size checks,\nwithout first-page HTTP validators. Failed pages resume from their saved URL.\nEach worker processes at most 100 pages, then schedules a continuation. Loops are\nrejected within a traversal batch. Settings saves restart from the first page.\nValidators are saved only for completed single-page polls; pending or failed\ntraversals cannot be hidden by a first-page HTTP 304.<\/p>\n Polling uses Consecutive feed-fetch or parse failures double the configured interval on each\nfailure, with a fixed 12-hour backoff cap. Neither that cap nor backoff can shorten\n HTTP 429\/503 responses honor The minute tick starts up to five background workers per site and Erlang node.\nEach worker polls up to five due feeds in sequence. Database leases prevent two\nworkers from polling the same feed concurrently. The overview shows article counts, cumulative import\/update\/skip\/error\nstatistics and poll times. Each feed has settings, pause\/resume, test\/preview, poll-now,\nreimport and paginated history for articles and dependent media.<\/p>\n Stable source URIs and checksums allow repeated polls without duplicates. Reimport\nprocesses only entries currently present in the feed, preserving resource IDs. Deleted\nresources remain deleted. Media already imported from the same URL is reused. Feeds\nwith more than 100 entries are rejected, rather than silently importing a partial feed.\nWebSub hub links are recognized in feed data; feed fetching uses polling.<\/p>\n Save & refetch deleted<\/strong> saves the settings and requests one fetch that can\nrestore deleted articles and their media when present in the current feed. It\nalso fetches a paused feed once, without resuming its regular polling. The request\nis consumed when a worker claims the feed, including if fetching fails; subsequent\npolls preserve deleted resources as usual. Missing entries are never restored.<\/p>\n Imports lock their source URIs across connected Erlang nodes, so independent feeds\ncan progress concurrently while shared articles and media are serialized. Network\nrequests and the complete article import do not hold a database transaction;\nresource and media APIs manage their own write transactions. The Deleting imported content checks each resource's delete permission and protection.\nDependent cleanup follows the normal resource deletion rules; a dependent does\nnot prevent deletion of its parent.<\/p>\n Discover Recurrence rules and recurrence IDs are retained as Zotonic calendar exports advertise Atom and RSS entries from With For ordinary feed entries, the Before writing an entry or its media, the importer dispatches the core\n Unchanged and deleted entries are skipped before notifying. Native feed entries\nnotify once; their delegation to mod_admin<\/code> and mod_cron<\/code>. Enter a website or feed URL, select one of the\ndiscovered feeds, and review three example entries beside the import settings.\nDiscovery uses HTML alternate links and HTTP Link headers.<\/p>\nuse<\/code> permission for mod_import_feed<\/code>. New feeds default to the current editor. Administrators and users with use<\/code> permission\nfor mod_admin_identity<\/code> can select another enabled account with Select user<\/strong>.\nSaving records that account and its user groups for background imports; ordinary\nsettings edits preserve the selected account and saved groups. The worker checks that the\neditor is still enabled and uses the editor's current, bounded permissions. Resource,\nmedia and edge operations enforce their normal ACLs; imported HTML is sanitized.<\/p>\nPer-feed settings<\/h2>\n
m_config<\/code> options.<\/p>\nSetting<\/th> Behavior<\/th><\/tr><\/thead> Import user<\/td> Defaults to the creator. Only administrators or users with full account access can select or change it.<\/td><\/tr> Default category<\/td> An existing non-meta category; initially Article.<\/td><\/tr> Content group<\/td> Available when mod_content_groups<\/code> is enabled. Empty uses the site default.<\/td><\/tr>Guess category<\/td> Match tags against existing category names or translated titles, case-insensitively; otherwise use the default category.<\/td><\/tr> Default connections<\/td> Use Add connection to select a predicate and page. Remove entries from the list as needed, then save. Existing article connections are preserved.<\/td><\/tr> Media<\/td> Dependent resources with depiction edges, the most important medium attached directly, or no media import.<\/td><\/tr> Fetch using<\/td> Anonymous by default, or registered OAuth2 credentials for the import user and destination host.<\/td><\/tr> Polling interval<\/td> 5 to 10080 minutes; initially 60 minutes.<\/td><\/tr> Authority<\/td> Non-authoritative by default; authoritative copies also retain source identities.<\/td><\/tr> WordPress enrichment<\/td> Enabled initially in the admin form; use the anonymous public REST API for content, author, dates and featured media. Failures fall back to feed data.<\/td><\/tr> Pause<\/td> Suspend automatic imports. New feeds are active unless saved paused.<\/td><\/tr><\/tbody><\/table>\n z-media<\/code> embeds referring to these stable resource IDs. After saving the\narticle and its connections, the importer queues persistent z_pivot_rsc<\/code> tasks\nfor the downloads. Attached media use the article itself as their placeholder.\nThe tasks use m_media<\/code> to identify each downloaded file and select its media\ncategory; downloads do not hold a database transaction.<\/p>\nRetry-After<\/code>\ntakes precedence. These limits are fixed, not module configuration. Permission failures stop immediately. The feed activity\nlog shows queued, started, retry, success and error events. A terminal failure leaves\nan empty placeholder so the article remains imported and a later reimport can retry.\nInline images can use data:<\/code> URLs, including base64 and percent-encoded data.\nThey follow the same deferred media import path and become z-media<\/code> embeds.\nA SHA-256 identity of the source URL keeps payloads out of resource URI indexes\nand activity logs, and lets repeated imports reuse the same resource. Embedded\nmedia have the same 20 MiB size limit as downloaded media.<\/p>\nsrcset<\/code> width or density, with src<\/code> as fallback. WordPress featured-image variants\nreuse the original image. In attached mode, other inline images retain their source URLs.<\/p>\nFetch authentication and development<\/h2>\n
z_fetch<\/code> routines;\nbrowser input cannot change the server's environment. OAuth2 credentials are never\nsent over HTTP. Anonymous fetching remains anonymous across redirects.<\/p>\nPolling and identity<\/h2>\n
ETag<\/code> and Last-Modified<\/code> after a successful\nimport and sends both validators on subsequent requests. HTTP 304 is a\nsuccessful unchanged poll: it does not parse a body, import entries, or remove\nexisting content, and preserves the publisher's polling interval. Validator\nupdates are protected by the poll lease and settings version. Failed imports,\nsettings changes, manual reimport and restoring deleted resources require a\nfresh response. Previews and individual-resource fetches remain unconditional.<\/p>\nrel=next<\/code> links and JSON Feed next_url<\/code> are followed. The initial\nimport reads only the first page unless Import all pages on the initial import<\/strong>\nis enabled (import_all_pages<\/code>, default false). This option can import large\namounts of articles and media.<\/p>\npoll_minutes<\/code> as a minimum interval (default 60 minutes; range\n5–10080). RSS ttl<\/code> and namespace-qualified syndication updatePeriod<\/code> \/\nupdateFrequency<\/code> hints can lengthen it. When both are present, the longer hint\nwins. Missing syndication frequency defaults to 1; a frequency without a period\nuses daily. Monthly\/yearly periods conservatively use 31\/366 days, and division\nrounds up to whole seconds. Invalid hints are ignored. Schedule alignment via\nupdateBase<\/code>, RSS skipHours<\/code>\/skipDays<\/code>, and HTTP cache freshness are not used.<\/p>\npoll_minutes<\/code> or the last valid publisher interval. A successful fetch and parse\nresets the failure counter and replaces the stored publisher hint, restoring the\nnormal frequency even if individual entries fail to import. Counters, publisher\nhints, and next-poll times survive restarts. There is no automatic give-up deadline.<\/p>\nRetry-After<\/code> as seconds or an HTTP date. The next\npoll waits for the longest of the normal interval, failure backoff, and that delay.\nPast dates mean no additional delay; malformed, duplicate or out-of-range values\nare ignored. Parsed delays must fit 0–2147483647 seconds. Entry-fetch errors can\nalso extend the next poll; media download retries honor the same header while\nretaining their sixteen-retry limit. Headers and response bodies are not\nsaved in error messages. Explicit Poll now<\/strong> remains an immediate override.\nNo new module configuration is required; poll_minutes<\/code> remains the per-feed control.<\/p>\nrsc_update<\/code>\nobserver writes ordinary feed resources' source mappings and feed membership in\nthat same transaction, including authoritative copies and media placeholders.\nCompletion checksums are recorded only after the entry finishes. A failed import can\nleave successfully imported resources in place for reuse on retry.<\/p>\niCalendar feeds<\/h2>\n
text\/calendar<\/code> links or enter an .ics<\/code> feed URL directly. VEVENTs\nmust have a UID, UTC DTSTAMP and valid DTSTART. Invalid events are skipped;\nmalformed calendar structure is rejected. Dates, durations and exclusive all-day\nend dates are normalized to Zotonic event properties. UTC and known IANA TZIDs\nare supported; floating times are interpreted as UTC. Custom VTIMEZONE rules\nare not evaluated, and events with unknown TZIDs are skipped. Cancelled events\nare unpublished. With category guessing enabled, calendar entries use event<\/code>;\notherwise the configured category applies.<\/p>\nical_rrule<\/code> and\nical_recurrence_id<\/code>; occurrences are not expanded. Exceptions have separate\nidentities derived from UID and RECURRENCE-ID. Alarms are ignored.<\/p>\nX-ZOTONIC-RESOURCE<\/code>, using the same native\nresource import and optional per-resource WebSub subscriptions as Atom\/RSS\/JSON\nFeed. Generic calendar feeds continue to use polling; feed-level WebSub push\nsubscriptions are not provided. X-ZOTONIC-NEXT<\/code> is followed using the existing\nfirst-page\/initial-import-all-pages and stop-at-seen-entry policies.<\/p>\nZotonic resource feeds<\/h2>\n
mod_export<\/code> advertise a JSON resource link with\nrelation https:\/\/zotonic.com\/rel\/resource<\/code>. JSON Feed items advertise the same\nendpoint in _zotonic.resource_url<\/code>. These use m_rsc_import<\/code>, preserving native\nproperties, categories, media embeds and connections. Connection import depth\ndefaults to 1 and accepts 0–10. Content-group\/copyright defaults and cleanup\npatterns apply; category guessing and media-layout choices do not reshape native\nresources. Choosing no media disables native medium downloads.<\/p>\nmod_websub<\/code> enabled, optional subscriptions cover imported articles and,\nwhen selected, their connections within the import depth. Both options default\nto false and require non-authoritative imports and a supporting source. Existing\nsubscriptions are managed separately through WebSub administration. Native\nresources use the core Fetch new version flow and saved import options, without\nrequiring the entry to remain in the feed. These refreshes use the core import\nnotification with native resource source metadata.<\/p>\nFetch new version<\/h2>\n
rsc_import_fetch<\/code> observer refreshes known imported articles from their current\nfeed, using the saved settings and import account. The caller needs feed-management\nand article-edit permission. Paused feeds can be refreshed manually without resuming\npolling. This bypasses unchanged checksums and still runs the import notification.\nAn article no longer present in the feed returns feed_entry_not_found<\/code>; unknown\nURIs fall through to the standard resource importer.<\/p>\nImport notification<\/h2>\n
#import_resource{item, related, source, options, raw}<\/code> notification. The source\nis #{importer => mod_import_feed, feed => Settings}<\/code>. Include\nzotonic_core\/include\/zotonic.hrl<\/code>, declare zotonic_observer<\/code>, and implement\nobserve_import_resource\/2<\/code> or its optional PID-aware variant. See the core\nobserver documentation for the payload, permissions and result contract.<\/p>\nm_rsc_import<\/code> preserves this scope, while\nrecursively imported resources may generate separate notifications.<\/p>","slug":"mod_import_feed","doc_module_config":[],"is_protected":false,"visible_for":0,"tz":"UTC","language":["en"],"doc_source_hash":"164a132a51a99d664afaba2bb2ef3352a89a6b166d17488cac8864fd4c8fe4ec","is_featured":false,"content_group_id":{"id":2551,"is_a":["meta","content_group"],"name":"content_group_imported_docs","title":"Imported documentation","uri":"https:\/\/zotonic.com\/id\/content_group_imported_docs"},"category_id":{"id":320,"is_a":["meta","category"],"name":"module","title":"Modules","uri":"https:\/\/test.zotonic.com\/id\/320"},"doc_source_path":"apps\/zotonic_mod_import_feed\/src\/mod_import_feed.erl","publication_start":"2026-10-13T15:15:42Z","github_url":"https:\/\/github.com\/zotonic\/zotonic\/blob\/master\/apps\/zotonic_mod_import_feed\/src\/mod_import_feed.erl","pivot_location_lng":null,"doc_source_kind":"module","name":"doc_module_mod_import_feed","is_unfindable":false,"is_published":true,"pivot_geocode":null,"created":"2026-10-13T15:15:42Z","uri":null,"doc_status":"current","is_dependent":false,"erlang_app":"zotonic_mod_import_feed","publication_end":"9999-06-01T00:00:00Z","modifier_id":{"id":1,"is_a":["person"],"name":"administrator","title":"Site Administrator","uri":"https:\/\/zotonic.com\/id\/1"},"privacy":0,"doc_source_commit":"7f5c498e5afc3d25f9c57b7bbf0b6435074ac952\n","creator_id":{"id":1,"is_a":["person"],"name":"administrator","title":"Site Administrator","uri":"https:\/\/zotonic.com\/id\/1"},"modified":"2026-11-03T14:49:05Z","title_slug":"mod_import_feed"},"uri":"https:\/\/zotonic.com\/id\/3085","uri_template":"https:\/\/zotonic.com\/id\/:id","websub":{"hub":"https:\/\/zotonic.com\/.zotonic\/websub","topic":"https:\/\/zotonic.com\/.zotonic\/websub\/topic\/3085"}},"status":"ok"}