Documentation · 06

Where the data comes from

Everything a persona says comes from text a real person wrote. This page is what that text is, and how to add your own.

The four kinds

Four kinds of source feed the corpus. /app/sources shows the coverage of each.

KindWhat it is
App reviewsApple App Store and Google Play reviews for the apps in the catalogue. One row per review, with a star rating.
Online communitiesPosts and comments about these apps and categories on Reddit, Hacker News and Bluesky. No star rating.
Published surveysToplines and segment breakdowns from published research such as Pew and the Reuters Institute Digital News Report.
Your uploadsA workspace's own CSVs: survey verbatims, support tickets, interview transcripts. Private to that workspace.

The public connectors are Reddit, Hacker News and Bluesky, all through public endpoints, plus the app stores. Uploads are pushed by you rather than polled. Anything you upload is private to your workspace.

Uploading a CSV

Upload a CSV or TSV of anything people wrote: survey verbatims, support tickets, interview transcripts, sales call notes. The upload form is at /app/sources, and it previews the file before anything is stored.

Columns

Only one column is required: the text. Headers are matched by name, case and punctuation ignored, against a list of common names:

FieldRequiredHeader names recognised
TextYestext, verbatim, comment, response, answer, open_response, feedback, review, body, message, notes, transcript, description, content, ticket_body, quote
RatingNorating, stars, star_rating, score, csat, satisfaction, nps, nps_score
DateNodate, created_at, submitted_at, timestamp, response_date, sent_at
AuthorNoauthor, respondent, user, customer, name, email, handle
TitleNotitle, subject, summary, headline, question
Source URLNosource_url, url, link, permalink, ticket_url
LanguageNolang, language, locale
CountryNocountry, country_code, market, region

If no header matches, userken falls back to the column with the longest average value, as long as that average is at least twenty characters. If it still cannot tell, it refuses the upload and lists your columns so you can name the right one yourself. Columns it does not recognise are kept alongside the row rather than discarded.

What happens to the values

  • Ratings are kept only when they are already 1 to 5. An NPS score out of 10 or a CSAT percentage is left blank rather than rescaled into something it is not, and the original value is kept beside the row.
  • Rows with no text are skipped.
  • Duplicates are collapsed on the text and its date, per upload, so re-uploading a file that grew does not double anything.
  • Missing dates default to the time of upload.
  • The delimiter is detected from comma, tab, semicolon or pipe.

An upload becomes an audience source in its own right, so you can filter and cluster it exactly like reviews. Files are read in memory and never written to a local disk. Uploads are part of the Team plan and above.

Survey pack formats

Published surveys are held as packs, so a synthetic panel can be asked the same question a real survey asked and scored against the published answer. Two formats.

YAML

source:
  publisher: Pew Research Center
  report: News Platform Fact Sheet 2025
  url: https://...
  methodology_url: https://...
  questionnaire_url: https://...
  year: 2025
  country: US
  sample_size: 5153
  population: US adults 18+
  method: online panel
  license: fair use, cited
questions:
  - id: pew2025_search_frequency
    question: "How often do you get news from search?"
    wording: exact (questionnaire NEWSPLAT_DIG, item SRCH)
    options: ["Often", "Sometimes", "Rarely", "Never"]
    toplines: {"Often": 19, "Sometimes": 44, "Rarely": 21, "Never": 15}
    ordinal: true
    audience_ref: cat:news
    segments:
      age:
        "18-29": {"Often": 31, "Sometimes": 40, "Rarely": 18, "Never": 11}

Long CSV

One row per question, segment and option. The survey and question metadata repeats on every row and the first non-empty value wins.

survey_id,publisher,report,year,question_id,question,option,share,segment_key,segment_value,n

question_id, question, option and share are required. segment_key and segment_value default to all, which is the topline. These optional columns are read when present: url, methodology_url, questionnaire_url, country, sample_size, population, method, license, notes, wording, audience_ref, ordinal, options.

Rules a pack has to pass

Shares are percentages, and a question's options have to sum to within a few points of 100. An ordinal question cannot carry a catch-all option such as don't know, because the distance measures assume the options are in order. A survey is identified by publisher, report and year, so re-ingesting the same survey replaces it rather than duplicating it.

Adding an app

The catalogue is the list of apps whose reviews userken holds, grouped into categories. Each app in it is automatically a built-in audience.

  1. Find the app. Search by name, or paste an App Store or Google Play URL, which resolves directly. userken tries to pair the Apple and Google listings for the same app.
  2. Add it with a display name and a category. A category that does not exist yet is created. You can set how many reviews to target per platform.
  3. Onboarding runs by itself and reports progress step by step: metadata, scraping both stores, saving, embedding, tagging themes, then building personas. When it finishes, the app goes active and pub:<slug> is a usable audience.

Adding apps is an operator action: it happens in the system admin area, or through the add_app tool with a key that is allowed to use it. Progress is visible while it runs, and the job is picked up by whichever host is free.

Was this page useful?Nothing is sent anywhere. This button only changes what you see.Thanks. Nothing was sent anywhere: this page has no analytics. If something here is wrong or missing, write to [email protected].