ClickPackageIndex

Differences between revisions 1 and 13 (spanning 12 versions)

Contents

Click Package Index

Click Package Index

Contact: James Tait (JamesTait)

Solr-backed repository for Click Application metadata
Public front-end will serve as access control, apply sane defaults and massage request and response for ease of use.
Interfaces with:
- Software Centre Agent
  - App developer defines app in Software Centre website
  - App developer uploads packaged app
  - Software Centre Agent pushes metadata to Click Package Index
    - First iteration, metadata is entered by developer
    - Later to be harvested from Click Manifest
- Dash
  - "Surfacing", i.e. first view of app lens before querying
  - "Search", i.e. list of apps that match search criteria
  - "Detail", i.e. full metadata for a given app

Solr Schema

Field Types

Name	Class	Notes
string	`solr.StrField`	Untokenised text.
boolean	`solr.BoolField`	True or False.
float	`solr.TrieFloatField`	Floating point number.
long	`solr.TrieLongField`	Long integer.
date	`solr.TrieDateField`	Date of the format `'YYYY-MM-DD"T"hh24:mm:ss[.nnn]"Z"'`, e.g. `'2013-06-27T12:39:45.776Z'`
url	`solr.StrField`	Untokenised text.
text_general	`solr.TextField`	Tokenised text. Reasonable cross-language defaults. `StandardTokenizer`, case-insensitive stop words, and down-casing. Query-time synonyms.
text_lang	`solr.TextField`	Localised tokenised text. `StandardTokenizer`, case-insensitive stop words, down-casing, protected words and minimal stemming. Query-time synonyms. See Localisation below.
key_value	`solr.TextField`	Key-value pair, i.e. `'key\|value'`.

Fields

Name	Type	Multi-value	Searchable	Retrievable	Required	Unique	Translatable
id	string	false	true	true	true	true	false
title	text_en	false	true	true	true	false	true
description	text_en	false	true	true	true	false	true
price	float	false	true	true	false	false	true (currencies)
name	text_general	false	true	true	true	true	false
binary_filesize	long	false	false	true	false	false	false
icon_url	url	false	false	true	false	false	false
icon_urls	payloads	true	true	true	false	false	false
screenshot_url	url	false	false	true	false	false	false
screenshot_urls	url	true	false	true	false	false	false
terms_of_service	url	false	false	true	false	false	true
support_url	url	false	false	true	false	false	true
license	text_general	false	true	true	true	false	false
date_published	date	false	true	true	false	false	false
video_urls	url	true	false	true	false	false	false
license_key_path	string	false	false	true	false	false	false
requires_license_key	boolean	false	true	true	false	false	false
version	string	false	true	true	true	false	false
website	url	false	false	true	false	false	true
company_name	text_general	false	true	true	false	false	false
keywords	text_en	true	true	true	false	false	true
click_version	string	false	true	true	true	false	false
framework	string	true	true	true	true	false	false
download_url	url	false	false	true	true	false	false
countries_to_distribute	string	true	true	false	false	false	false
text	text_en	true	true	false	false	false	true

Localisation

Fields marked as translatable will have localised variants whose field name is suffixed with one of the following country codes:

Suffix	Language
ar	Arabic
bg	Bulgarian
ca	Catalan
cn	Chinese
cz	Czech
da	Danish
de	German
el	Greek
en	English
es	Spanish
eu	Basque
fa	Persian
fi	Finnish
fr	French
ga	Irish
gl	Galician
hi	Hindi
hu	Hungarian
hy	Armenian
id	Indonesian
it	Italian
ja	Japanese
kr	Korean
lv	Latvian
nl	Dutch
no	Norwegian
pt	Portuguese
ro	Romanian
ru	Russian
sv	Swedish
th	Thai
tr	Turkish

For queries:

If the query specifies a localisation (e.g. q=description_en:foo), we should use it.
Similarly, if a specific localised result field is requested (e.g. fl=description_en), we should use it.
In the general case, clients should use the non-localised field name in queries (q=description) and result fields (fl=description) and provide an Accept-Language header, which we will use to choose the localised variant.
If the request specifies an alternative localisation (lang=en) we should use that in preference to the language in the Accept-Language header.

This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case. We'll need to be wary of caching, in particular - Vary: Accept-Language and ETag headers will be required at least.

API

Guiding principles:

JSON-based
Discoverable
Browseable

API Root

/api/v1

Provides clickable links to the other endpoints for discoverability.

Request

GET /api/v1 HTTP/1.1
Host: search.apps.ubuntu.com

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
    "search": "http://search.apps.ubuntu.com/api/v1/search",
    "package": "http://search.apps.ubuntu.com/api/v1/package"
}

Search

/api/v1/search

Proxies requests to Solr's SearchHandler. A subset of the standard Solr syntax is used for querying.

Request

GET /api/v1/search?q=click_framework:ubuntu-sdk-13.10,description:awesome&wt=json&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"click_framework:ubuntu-sdk-13.10,description:awesome",
      "wt":"json",
      "version":"2.2",
      "rows":"10"
    }
  },
  "response":{
    "numFound":2,
    "start":0,
    "docs":[
      {
        "id": "1",
        "title": "Awesome Launcher",
        "description": "This is an awesome launcher.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/1"
      },
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/2"
      }
    ]
  }
}

Details

/api/v1/package

Requests details for a specific package. Cleans the query string to ensure the user cannot be tricked into installing the wrong package, and enforces a single item in the response.

Request

GET /api/v1/package/2?wt=json&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"id:2",
      "wt":"json",
      "version":"2.2",
      "rows":"1"
    }
  },
  "response":{
    "numFound":1,
    "start":0,
    "docs":[
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
      }
    ]
  }
}

AppStore/Interfaces/ClickPackageIndex (last edited 2018-04-01 09:09:46 by popey)

-  ⇤ ← Revision 1 as of 2013-06-21 13:46:35 → 
  Size: 54
  Editor: 173-165-137-65-utah
  Comment:
+   ← Revision 13 as of 2013-06-28 11:44:29 → ⇥
  Size: 11497
  Editor: sixtymilesmilew
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-Describe AppStore/Interfaces/ClickPackageIndex here.
+<<TableOfContents(3)>>

= Click Package Index =
''Contact: James Tait (JamesTait)''

 * Solr-backed repository for Click Application metadata
 * Public front-end will serve as access control, apply sane defaults and massage request and response for ease of use.
 * Interfaces with:
  * Software Centre Agent
   * App developer defines app in Software Centre website
   * App developer uploads packaged app
   * Software Centre Agent pushes metadata to Click Package Index
    * First iteration, metadata is entered by developer
    * Later to be harvested from Click Manifest
  * Dash
   * "Surfacing", i.e. first view of app lens before querying
   * "Search", i.e. list of apps that match search criteria
   * "Detail", i.e. full metadata for a given app


== Solr Schema ==

=== Field Types ===

|| '''Name'''   || '''Class'''         || '''Notes''' ||
|| string       || {{{solr.StrField}}}       || Untokenised text. ||
|| boolean      || {{{solr.BoolField}}}      || True or False. ||
|| float        || {{{solr.TrieFloatField}}} || Floating point number. ||
|| long         || {{{solr.TrieLongField}}}  || Long integer. ||
|| date         || {{{solr.TrieDateField}}}  || Date of the format {{{'YYYY-MM-DD"T"hh24:mm:ss[.nnn]"Z"'}}}, e.g. {{{'2013-06-27T12:39:45.776Z'}}} ||
|| url          || {{{solr.StrField}}}       || Untokenised text. ||
|| text_general || {{{solr.TextField}}}      || Tokenised text. Reasonable cross-language defaults. {{{StandardTokenizer}}}, case-insensitive stop words, and down-casing.  Query-time synonyms. ||
|| text_''lang''|| {{{solr.TextField}}}      || Localised tokenised text. {{{StandardTokenizer}}}, case-insensitive stop words, down-casing, protected words and minimal stemming.  Query-time synonyms. See [[#Localisation|Localisation]] below.||
|| key_value    || {{{solr.TextField}}}      || Key-value pair, i.e. {{{'key|value'}}}. ||

=== Fields ===

|| '''Name'''              || '''Type'''   || '''Multi-value''' || '''Searchable''' || '''Retrievable''' || '''Required''' || '''Unique''' || '''Translatable''' ||
|| id                      || string       || false             || true             || true              || true           || true         || false        ||
|| title                   || text_en      || false             || true             || true              || true           || false        || true         ||
|| description             || text_en      || false             || true             || true              || true           || false        || true         ||
|| price                   || float        || false             || true             || true              || false          || false        || true (currencies)     ||
|| name                    || text_general || false             || true             || true              || true           || true         || false        ||
|| binary_filesize         || long         || false             || false            || true              || false          || false        || false        ||
|| icon_url                || url          || false             || false            || true              || false          || false        || false        ||
|| icon_urls               || payloads     || true              || true             || true              || false          || false        || false        ||
|| screenshot_url          || url          || false             || false            || true              || false          || false        || false        ||
|| screenshot_urls         || url          || true              || false            || true              || false          || false        || false        ||
|| terms_of_service        || url          || false             || false            || true              || false          || false        || true         ||
|| support_url             || url          || false             || false            || true              || false          || false        || true         ||
|| license                 || text_general || false             || true             || true              || true           || false        || false         ||
|| date_published          || date         || false             || true             || true              || false          || false        || false        ||
|| video_urls              || url          || true              || false            || true              || false          || false        || false        ||
|| license_key_path        || string       || false             || false            || true              || false          || false        || false        ||
|| requires_license_key    || boolean      || false             || true             || true              || false          || false        || false        ||
|| version                 || string       || false             || true             || true              || true           || false        || false        ||
|| website                 || url          || false             || false            || true              || false          || false        || true         ||
|| company_name            || text_general || false             || true             || true              || false          || false        || false        ||
|| keywords                || text_en      || true              || true             || true              || false          || false        || true         ||
|| click_version           || string       || false             || true             || true              || true           || false        || false        ||
|| framework               || string       || true              || true             || true              || true           || false        || false        ||
|| download_url            || url          || false             || false            || true              || true           || false        || false        ||
|| countries_to_distribute || string       || true              || true             || false             || false          || false        || false        ||
|| text                    || text_en      || true              || true             || false             || false          || false        || true         ||

== Localisation ==

Fields marked as translatable will have localised variants whose field name is suffixed with one of the following country codes:

|| '''Suffix''' || '''Language''' ||
|| ar           || Arabic         ||
|| bg           || Bulgarian      ||
|| ca           || Catalan        ||
|| cn           || Chinese        ||
|| cz           || Czech          ||
|| da           || Danish         ||
|| de           || German         ||
|| el           || Greek          ||
|| en           || English        ||
|| es           || Spanish        ||
|| eu           || Basque         ||
|| fa           || Persian        ||
|| fi           || Finnish        ||
|| fr           || French         ||
|| ga           || Irish          ||
|| gl           || Galician       ||
|| hi           || Hindi          ||
|| hu           || Hungarian      ||
|| hy           || Armenian       ||
|| id           || Indonesian     ||
|| it           || Italian        ||
|| ja           || Japanese       ||
|| kr           || Korean         ||
|| lv           || Latvian        ||
|| nl           || Dutch          ||
|| no           || Norwegian      ||
|| pt           || Portuguese     ||
|| ro           || Romanian       ||
|| ru           || Russian        ||
|| sv           || Swedish        ||
|| th           || Thai           ||
|| tr           || Turkish        ||

For queries:

 * If the query specifies a localisation (e.g. {{{q=description_en:foo}}}), we should use it.
 * Similarly, if a specific localised result field is requested (e.g. {{{fl=description_en}}}), we should use it.
 * In the general case, clients should use the non-localised field name in queries ({{{q=description}}}) and result fields ({{{fl=description}}}) and provide an {{{Accept-Language}}} header, which we will use to choose the localised variant.
 * If the request specifies an alternative localisation ({{{lang=en}}}) we should use that in preference to the language in the {{{Accept-Language}}} header.

This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case.  We'll need to be wary of caching, in particular - {{{Vary: Accept-Language}}} and {{{ETag}}} headers will be required at least.

== API ==
Guiding principles:

 * JSON-based
 * Discoverable
 * Browseable

=== API Root ===
`/api/v1`

Provides clickable links to the other endpoints for discoverability.

==== Request ====
{{{
GET /api/v1 HTTP/1.1
Host: search.apps.ubuntu.com
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
    "search": "http://search.apps.ubuntu.com/api/v1/search",
    "package": "http://search.apps.ubuntu.com/api/v1/package"
}
}}}

=== Search ===
`/api/v1/search`

Proxies requests to Solr's {{{SearchHandler}}}.  A subset of the standard Solr syntax is used for querying.

==== Request ====
{{{
GET /api/v1/search?q=click_framework:ubuntu-sdk-13.10,description:awesome&wt=json&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"click_framework:ubuntu-sdk-13.10,description:awesome",
      "wt":"json",
      "version":"2.2",
      "rows":"10"
    }
  },
  "response":{
    "numFound":2,
    "start":0,
    "docs":[
      {
        "id": "1",
        "title": "Awesome Launcher",
        "description": "This is an awesome launcher.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/1"
      },
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/2"
      }
    ]
  }
}
}}}

=== Details ===
`/api/v1/package`

Requests details for a specific package.  Cleans the query string to ensure the
user cannot be tricked into installing the wrong package, and enforces a single
item in the response.

==== Request ====
{{{
GET /api/v1/package/2?wt=json&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"id:2",
      "wt":"json",
      "version":"2.2",
      "rows":"1"
    }
  },
  "response":{
    "numFound":1,
    "start":0,
    "docs":[
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
      }
    ]
  }
}
}}}

Ubuntu Wiki