Skip to main content

Firehose data format

There’s a standard file format required for sending browse and transactional data from Fresh Relevance to external systems.

Updated over a week ago

Data is normally exported by uploading files at regular intervals to an FTP server. Fresh Relevance supports three versions - FTP, SFTP, and FTPES. Your technical support team should know which you need.

Some typical examples of usage are:

  • Exporting browse and transactional data to a CRM platform or ESP for marketing or other communication purposes.

  • Exporting to a marketing analytics system/SCV.

The rate of exporting files can be set to a value in minutes. For example, every 10 minutes or every day. By default, the system uploads a file every 30 minutes.


File format

A succession of flat files are exported, with names corresponding to the date-time.

The files contain CSV data, with one record per line item. So, for example, if a cart containing five products is purchased there will be five purchase events, each for one product. The values which don't change with the line item are repeated on each record.

All data is optional and can be mapped to field names of your choice.

Standard signal types

The following types of signals can be enabled as standard for output in Firehose data.

Contact Support to check which signals are being generated in your account and which are enabled for output in Firehose.

Signal type

Signal name

Signal description

pd

Product Browse

Product details page visit.

c

Cart Visit

Cart page visit.

p

Checkout Visit

Checkout page visit.

t

Purchase Complete

Shopper has completed a purchase.

ba

Browse Abandon

Shopper has abandoned a browse session with an empty cart.

b

Cart Abandon

Shopper has abandoned a cart.

cu

Custom

Generated for other page types or interactions, for example, a wishlist page visit.

se

Session Expired

Browsing session expires.

pcv

Price Drop

Fired for every person included in a pricedrop segment.

pcs

Back in Stock

Fired for every person included in a back-in-stock segment.

pcls

Low stock

A product is low in stock.

pi

Person Identified

A new person has been identified.

rt

Returned Purchase

Any product is returned.

cc

Cart Changed

Shopper adds or removes products from their cart.

Attribution signal types

The following types of signals are generated for internal reporting purposes and are not likely to be useful in Firehose data. However they can be enabled for Firehose output if you want.

Contact Support to enable these.

Signal type

Signal name

Signal description

ia

Internal Attribution

When a purchase is attributed to onsite content.

ea

External Attribution

When a purchase is attributed to offsite content.

ESP signal types

The following types of signals can only be generated for supported ESPs. They allow the transfer of email based events to Fresh Relevance.

Currently only Acoustic's UBX platform is supported. Contact Support if you want to enable these for output in Firehose.

Signal type

Signal name

Signal description

es

Email Send

Signal fired when an external service reports an email send.

eo

Email Open

Signal fired when an external service reports an email open.

ec

Email Click

Signal fired when an external service reports an email clickthrough.

cs

Click Stream

Signal fired when an external service reports an email clickstream event.

email_attachment

Email Attachment

Signal fired when an external service reports a recipient has opened an email attachment.

ebl

Email Block

Signal fired when an external service reports an email has been blocked.

ebo

Email Bounce

Signal fired when an external service reports an email has bounced including the type (soft/hard).

eco

Email Conversion

Signal fired when an external service reports an email conversion.

eoo

Email Opt-Out

Signal fired when an external service reports a contact has opted out through an email unsubscribe link.

er

Email Restriction

Signal fired when an external service reports an email send has been restricted.

esu

Email Suppressed

Signal fired when an external service reports an email send has been suppressed.

rae

Email Reply Abuse

Signal fired when an external service reports an email has been reported for abuse.

ro

Email Reply Other

Signal fired when an external service reports a generic reply event.


Data

Common data:

  • Email address

  • First name

  • Last name

  • Any available field from the Person record

  • Date

  • Signal ID - the ID unique to each signal. Where multiple products are present in the signal, the same signal ID is used for multiple records (V2 only).

  • Unique ID - the ID unique to each record exported. This is based on the signal ID, but has the row number appended for signals containing multiple products (V2 only).

Additional data for Product Browsed records:

  • Item product ID (item_prid)

  • Item Price (item_up)

  • Regular item price

  • Item tax

  • CSV-delimited (,) list of categories for the product. The field is enclosed in double quotes.

  • Reviews

  • product_opt — Variable data selected by user. For example, shoe size, product.opt.size = 9.

  • product_ex — Fixed custom product data specific to a client's account, for example, brand, product.ex.brand. It should always be the same value.

Additional data for Carted or Abandoned records:

  • Campaign source

  • Campaign ID

  • Campaign link

  • cart_price

  • Shipping

Additional data for Purchase records:

  • Order date

  • order_id

    The same for all products in an order.

  • Item product ID (item_prid)

  • Item price (item_up)

  • Item name (item_n)

  • Quantity (item_qty)

  • cart_price

  • Shipping

  • Campaign source

  • Campaign ID

  • Campaign link

Additional data for Session Expired (SE) records:

  • Session length in seconds (signal.len_secs)

  • Session first start date/time (signal.fdt)

  • Session expiry date/time (signal.edt)

  • Session last activity date/time (signal.ldt)

  • First URL (signal.first_url)

  • Last URL (signal.last_url)

  • Signals in this session (signals)

  • Number of product details signals in this session (signals.pd|length)

  • Number of purchase complete signals in this session (signals.t|length)

  • Number of custom signals in this session (signals.cu|length)

  • Number of cart abandon signals in this session (signals.b|length)

  • Number of browse abandon signals in this session (signals.ba|length)

  • Number of cart signals in this session (signals.c|length)

Date/times are normally output in ISO format, for example, 2018-02-22 13:23:03. For integration purposes, it's sometimes useful to format the datetime in w3c format, for example, 2018-02-22T13:23:03.123221+00:00.

To do this, apply a filter to the field mapping, like this:

{{ signal.fdt | datetime_w3c }}

Product interaction data

Information about the products browsed or clicked during a session is also passed in Session Expired records, but only for channels other than FTP.

Additional data for product interaction:

  • SmartBlock seen (person_product_seen)

  • SmartBlock clicked (person_product_clicked)

These data collections include the product ID, Slot ID, and SmartBlock ID for each product that a visitor has interacted with.

For click data to be successfully exported, you must use the default JSON schema.

Did this answer your question?