ClickPackageIndex

Differences between revisions 16 and 17

Contents

Click Package Index

Click Package Index

Contact: James Tait (JamesTait)

Solr-backed repository for Click Application metadata
Public front-end will serve as access control, apply sane defaults and massage request and response for ease of use.
Interfaces with:
- Software Centre Agent
  - App developer defines app in Software Centre website
  - App developer uploads packaged app
  - Software Centre Agent pushes metadata to Click Package Index
    - First iteration, metadata is entered by developer
    - Later to be harvested from Click Manifest
- Dash
  - "Surfacing", i.e. first view of app lens before querying
  - "Search", i.e. list of apps that match search criteria
  - "Detail", i.e. full metadata for a given app

Solr Schema

Field Types

Name	Class	Notes
string	`solr.StrField`	Untokenised text.
boolean	`solr.BoolField`	True or False.
float	`solr.TrieFloatField`	Floating point number.
long	`solr.TrieLongField`	Long integer.
date	`solr.TrieDateField`	Date of the format `'YYYY-MM-DD"T"hh24:mm:ss[.nnn]"Z"'`, e.g. `'2013-06-27T12:39:45.776Z'`
url	`solr.StrField`	Untokenised text.
text_general	`solr.TextField`	Tokenised text. Reasonable cross-language defaults. `StandardTokenizer`, case-insensitive stop words, and down-casing. Query-time synonyms.
text_lang	`solr.TextField`	Localised tokenised text. `StandardTokenizer`, case-insensitive stop words, down-casing, protected words and minimal stemming. Query-time synonyms. See Localisation below.
key_value	`solr.TextField`	Key-value pair, i.e. `'key\|value'`.

Fields

Name	Type	Multi-value	Searchable	Retrievable	Required	Unique	Translatable
id	string	false	true	true	true	true	false
title	text_en	false	true	true	true	false	true
description	text_en	false	true	true	true	false	true
price	float	false	true	true	false	false	true (currencies)
name	text_general	false	true	true	true	true	false
binary_filesize	long	false	false	true	false	false	false
icon_url	url	false	false	true	false	false	false
icon_urls	payloads	true	true	true	false	false	false
screenshot_url	url	false	false	true	false	false	false
screenshot_urls	url	true	false	true	false	false	false
terms_of_service	url	false	false	true	false	false	true
support_url	url	false	false	true	false	false	true
license	text_general	false	true	true	true	false	false
date_published	date	false	true	true	false	false	false
video_urls	url	true	false	true	false	false	false
license_key_path	string	false	false	true	false	false	false
requires_license_key	boolean	false	true	true	false	false	false
version	string	false	true	true	true	false	false
website	url	false	false	true	false	false	true
company_name	text_general	false	true	true	false	false	false
keywords	text_en	true	true	true	false	false	true
click_version	string	false	true	true	true	false	false
framework	string	true	true	true	true	false	false
download_url	url	false	false	true	true	false	false
countries_to_distribute	string	true	true	false	false	false	false
text	text_en	true	true	false	false	false	true

Localisation

Fields marked as translatable will have localised variants whose field name is suffixed with one of the following country codes:

Suffix	Language
ar	Arabic
bg	Bulgarian
ca	Catalan
cn	Chinese
cz	Czech
da	Danish
de	German
el	Greek
en	English
es	Spanish
eu	Basque
fa	Persian
fi	Finnish
fr	French
ga	Irish
gl	Galician
hi	Hindi
hu	Hungarian
hy	Armenian
id	Indonesian
it	Italian
ja	Japanese
kr	Korean
lv	Latvian
nl	Dutch
no	Norwegian
pt	Portuguese
ro	Romanian
ru	Russian
sv	Swedish
th	Thai
tr	Turkish

For queries:

If the query specifies a localisation (e.g. q=description_en:foo), we will use it.
Similarly, if a specific localised result field is requested (e.g. fl=description_en), we will use it.
In the general case, clients should use the non-localised field name in queries (q=description) and result fields (fl=description) and provide an Accept-Language header, which we will use to choose the localised variant.
If the request specifies an alternative localisation via the lang query parameter (e.g. lang=en) we will use that in preference to the language in the Accept-Language header.
If the request specifies a localisation which is unavailable, the English localisation will be returned.

This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case. We'll need to be wary of caching, in particular - Vary: Accept-Language and ETag headers will be required at least.

API

Guiding principles:

JSON-based
Discoverable
Browseable

API Root

/api/v1

Provides clickable links to the other endpoints for discoverability.

Request

GET /api/v1 HTTP/1.1
Host: search.apps.ubuntu.com

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
    "search": "http://search.apps.ubuntu.com/api/v1/search",
    "package": "http://search.apps.ubuntu.com/api/v1/package"
}

Search

/api/v1/search

Proxies requests to Solr's SearchHandler. A subset of the standard Solr syntax is used for querying. Clients should indicate the desired localisation as described in Localisation.

Request

GET /api/v1/search?q=framework:ubuntu-sdk-13.10,description:awesome&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"click_framework:ubuntu-sdk-13.10,description:awesome",
      "wt":"json",
      "version":"2.2",
      "rows":"10"
    }
  },
  "response":{
    "numFound":2,
    "start":0,
    "docs":[
      {
        "id": "1",
        "title": "Awesome Launcher",
        "description": "This is an awesome launcher.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/1"
      },
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
        "resource_url": "http://search.apps.ubuntu.com/api/v1/package/2"
      }
    ]
  }
}

Details

/api/v1/package

Requests details for a specific package. Cleans the query string to ensure the user cannot be tricked into installing the wrong package, and enforces a single item in the response.

Request

GET /api/v1/package/2?wt=json&fl=id,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: xyz987

{
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "fl": "id,title,description,price,icon_url",
      "indent":"on",
      "start":"0",
      "q":"id:2",
      "wt":"json",
      "version":"2.2",
      "rows":"1"
    }
  },
  "response":{
    "numFound":1,
    "start":0,
    "docs":[
      {
        "id": "2",
        "title": "Awesome Widget",
        "description": "This is an awesome widget.",
        "price": 1.99,
        "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
      }
    ]
  }
}

AppStore/Interfaces/ClickPackageIndex (last edited 2018-04-01 09:09:46 by popey)

-  ⇤ ← Revision 16 as of 2013-07-01 11:35:49 → 
  Size: 11755
  Editor: sixtymilesmilew
  Comment: Schoolboy error - we don't want to search for documents based on the localised URL, we want to see the localised URL for the search result
+   ← Revision 17 as of 2013-07-03 18:04:59 → ⇥
  Size: 11800
  Editor: sixtymilesmilew
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 106:
- * If the query specifies a localisation (e.g. {{{q=description_en:foo}}}), we will use it.
 * Similarly, if a specific localised result field is requested (e.g. {{{fl=description_en}}}), we will use it.
 * In the general case, clients should use the non-localised field name in queries ({{{q=description}}}) and result fields ({{{fl=description}}}) and provide an {{{Accept-Language}}} header, which we will use to choose the localised variant.
 * If the request specifies an alternative localisation via the {{{lang}}} query parameter ({{{lang=en}}}) we will use that in preference to the language in the {{{Accept-Language}}} header.
+ * If the query specifies a localisation (e.g. `q=description_en:foo`), we will use it.
 * Similarly, if a specific localised result field is requested (e.g. `fl=description_en`), we will use it.
 * In the general case, clients should use the non-localised field name in queries (`q=description`) and result fields (`fl=description`) and provide an `Accept-Language` header, which we will use to choose the localised variant.
 * If the request specifies an alternative localisation via the `lang` query parameter (e.g. `lang=en`) we will use that in preference to the language in the `Accept-Language` header.
 Line 112:
-This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case.  We'll need to be wary of caching, in particular - {{{Vary: Accept-Language}}} and {{{ETag}}} headers will be required at least.
+This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case.  We'll need to be wary of caching, in particular - `Vary: Accept-Language` and `ETag` headers will be required at least.
 Line 145:
-Proxies requests to Solr's {{{SearchHandler}}}.  A subset of the standard Solr syntax is used for querying.
+Proxies requests to Solr's `SearchHandler`.  A subset of the standard Solr syntax is used for querying.  Clients should indicate the desired localisation as described in [[#Localisation|Localisation]].
 Line 149:
-GET /api/v1/search?q=click_framework:ubuntu-sdk-13.10,description:awesome&wt=json&fl=id,title,description,price,icon_url HTTP/1.1
+GET /api/v1/search?q=framework:ubuntu-sdk-13.10,description:awesome&fl=id,title,description,price,icon_url HTTP/1.1

Ubuntu Wiki