ClickPackageIndex

Revision 34 as of 2014-04-02 17:00:33

Clear message

Contents

Click Package Index

Click Package Index

Contact: James Tait (JamesTait)

Lucene-backed repository for Click Application metadata
Public front-end will serve as access control, apply sane defaults and massage request and response for ease of use.
Interfaces with:
- Software Centre Agent
  - App developer defines app in Software Centre website
  - App developer uploads packaged app
  - Software Centre Agent pushes metadata to Click Package Index
    - Metadata harvested from Click Manifest
    - Verified and corrected by developer
- Dash
  - "Surfacing", i.e. first view of app lens before querying
  - "Search", i.e. list of apps that match search criteria
  - "Detail", i.e. full metadata for a given app

Schema

Field Types

Name	Class	Notes
string	`solr.StrField`	Untokenised text.
boolean	`solr.BoolField`	True or False.
float	`solr.TrieFloatField`	Floating point number.
long	`solr.TrieLongField`	Long integer.
date	`solr.TrieDateField`	Date of the format `'YYYY-MM-DD"T"hh24:mm:ss[.nnn]"Z"'`, e.g. `'2013-06-27T12:39:45.776Z'`
url	`solr.StrField`	Untokenised text.
text_general	`solr.TextField`	Tokenised text. Reasonable cross-language defaults. `StandardTokenizer`, case-insensitive stop words, and down-casing. Query-time synonyms.
text_lang	`solr.TextField`	Localised tokenised text. `StandardTokenizer`, case-insensitive stop words, down-casing, protected words and minimal stemming. Query-time synonyms. See Localisation below.
key_value	`solr.TextField`	Key-value pair, i.e. `'key\|value'`.
category	`solr.TextField`	Tokenised text, with a semi-colon delimiter and down-casing.

Fields

Name	Type	Multi-value	Searchable	Retrievable	Required	Translatable	Notes
name	string	false	true	true	true	false	e.g. org.example.my_app
version	string	false	true	true	true	false
click_version	string	false	true	true	true	false
framework	string	true	true	true	true	false	Deprecated This field should not be used as a search parameter. The list of supported frameworks should be passed as a comma-separated list in the X-Ubuntu-Frameworks header instead. If both the header and the parameter are present, the header will take precedence.
license	text_general	false	true	true	true	false	License under which the software is made available.
download_url	url	false	false	true	true	false
title_lang	text_lang	false	true	true	true (en)	true	Display name, as presented in menus.
description_lang	text_lang	false	true	true	true (en)	true	Full description of the app's functionality.
changelog	text_general	false	true	true	false	false	List of changes in this version.
text_lang	text_lang	true	true	false	false	true	Default searchable field containing tokens from title, description and keywords.
keywords_lang	text_lang	true	true	true	false	true
price	float	false	true	true	false	true (currencies)
support_url_lang	url	false	false	true	false	true	URL to localised support channel.
terms_of_service_lang	url	false	false	true	false	true	URL to localised ToS.
website_lang	url	false	false	true	false	true
binary_filesize	long	false	false	true	false	false	Size of the downloadable package, in bytes.
icon_url	url	false	false	true	false	false	URL to a 64x64px PNG image to use as an application icon, e.g. in menus.
icon_urls	string	true	true	true	false	false	List of delimited strings in the format ${size}\|${url} describing URLs to square PNG icons of various sizes.
screenshot_url	url	false	false	true	false	false	URL to a screen shot.
screenshot_urls	url	true	false	true	false	false	List of URLs to other screenshots.
company_name	text_general	false	true	true	false	false
developer_name	text_general	false	true	true	false	false	The full name of the developer profile which uploaded the package.
publisher	text_general	false	true	true	false	false	The company name or the full name of the developer profile which uploaded the package if the company name is not available.
date_published	date	false	true	true	false	false	Date of publication.
video_urls	url	true	false	true	false	false
license_key_path	string	false	false	true	false	false
requires_license_key	boolean	false	true	true	false	false
whitelist_country_codes	string	true	true	false	false	false
blacklist_country_codes	string	true	true	false	false	false
prices	string	true	true	true	false	false
architecture	string	true	true	true	false	false	Deprecated This field should not be used as a search parameter. The client architecture should be passed in the X-Ubuntu-Architecture header instead. If both the header and the parameter are present, the header will take precedence.
category	category	false	true	true	false	false	See the Software Centre Genre Spec and the freedesktop.org Desktop Menu Spec
department	string	true	true	true	false	false	See the Software Centre Genre Spec and the freedesktop.org Desktop Menu Spec
highlights	key_value	true	true	true	false	false

Localisation

Fields marked as translatable will have localised variants whose field name is suffixed with one of the following country codes:

Suffix	Language
ar	Arabic
az	Azerbaijani
bg	Bulgarian
bn	Bengali
bs	Bosnian
ca	Catalan
cs	Czech
cy	Welsh
da	Danish
de	German
el	Greek
en	English
en-gb	British English
eo	Esperanto
es	Spanish
es-ar	Argentine Spanish
es-mx	Mexican Spanish
es-ni	Nicaraguan Spanish
et	Estonian
eu	Basque
fa	Persian
fi	Finnish
fr	French
fy-nl	Frisian
ga	Irish
gl	Galician
he	Hebrew
hi	Hindi
hr	Croatian
hu	Hungarian
id	Indonesian
is	Icelandic
it	Italian
ja	Japanese
ka	Georgian
kk	Kazakh
km	Khmer
kn	Kannada
ko	Korean
lt	Lithuanian
lv	Latvian
mk	Macedonian
ml	Malayalam
mn	Mongolian
nb	Norwegian Bokmål
ne	Nepali
nl	Dutch
nn	Norwegian Nynorsk
pa	Punjabi
pl	Polish
pt	Portuguese
pt-br	Brazilian Portuguese
ro	Romanian
ru	Russian
sk	Slovak
sl	Slovenian
sq	Albanian
sr	Serbian
sr-latn	Serbian Latin
sv	Swedish
sw	Swahili
ta	Tamil
te	Telugu
th	Thai
tr	Turkish
tt	Tatar
uk	Ukrainian
ur	Urdu
vi	Vietnamese
zh-cn	Simplified Chinese
zh-tw	Traditional Chinese

For queries:

If the query specifies a localisation (e.g. q=description_en:foo), we will use it.
Similarly, if a specific localised result field is requested (e.g. fields=description_en), we will use it.
In the general case, clients should use the non-localised field name in queries (q=description) and result fields (fields=description) and provide an Accept-Language header, which we will use to choose the localised variant.
If the request specifies an alternative localisation via the lang query parameter (e.g. lang=en) we will use that in preference to the language in the Accept-Language header.
If the request specifies a localisation which is unavailable, the English localisation will be returned.

This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case. We'll need to be wary of caching, in particular - Vary: Accept-Language and ETag headers will be required at least.

Client Platform Properties

The client's architecture and supported frameworks are platform properties - the user isn't going to enter these in the dash, they're intrinsic to the device and aren't going to change from one search to the next. They're also not strict requirements - the intended meaning is "this device supports these features", not "all of these features must be required by the package". For example, a search for

keywords:maps,title:google

should return Google Maps, but not Bing Maps. On the other hand, a search for

architecture:armhf

isn't saying "only give me packages for armhf", but "my architecture is armhf - do the hard work for me and discard anything I can't use", and would also return packages with architecture:all; and a search specifying

framework:ubuntu-sdk-13.10,framework:ubuntu-sdk-14.04

isn't expecting to find packages that require *both* frameworks, but is instead saying "I support both of these and nothing else - do the hard work for me and discard anything I can't use". They're not strict requirements on the package - all conditions don't have to be met - but a statement about the execution environment. The search is additive, i.e. specifying more values for these properties should *add* results, not take them away.

These properties will be passed as request headers, similarly to the user's language setting, as outlined below.

Client Architecture

The client's architecture will be passed as a single string in the X-Ubuntu-Architecture request header. Recognised values are: armhf, i386, x86_64. The value provided will be combined combined with the value all by the server, such that the result will include packages that specifically target the client architecture, and those that have no architecture-specific components. For example, the request header:

X-Ubuntu-Architecture: armhf

would match the following packages:

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "architecture": ["armhf", "i386"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "architecture": ["all"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

but not the following:

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "architecture": ["i386"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "architecture": ["x86_64"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

Frameworks

The frameworks supported by the client will be passed as a comma-separated list of values in the the X-Ubuntu-Frameworks request header. The values provided will be used in an exact match query, such that the frameworks listed by packages in the result will be a subset of those provided by the client. For example, the request header:

X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04

would match the following packages:

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "framework": ["ubuntu-sdk-13.10"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "framework": ["ubuntu-sdk-14.04"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

but not:

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "framework": ["ubuntu-sdk-13.10", "fictional-framework-13.10"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "framework": ["ubuntu-sdk-14.04-dev"],
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

API

Guiding principles:

JSON-based
Discoverable
Browseable

API Root

/api/v1

Provides clickable links to the other endpoints for discoverability.

Request

GET /api/v1 HTTP/1.1
Host: search.apps.ubuntu.com
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
    "search": "http://search.apps.ubuntu.com/api/v1/search",
    "package": "http://search.apps.ubuntu.com/api/v1/package",
    "highlights": "http://search.apps.ubuntu.com/api/v1/highlights",
    "departments": "http://search.apps.ubuntu.com/api/v1/departments"
}

Search

/api/v1/search

Search the index for packages. Search terms may be entered unqualified, in which case the search will be performed against the localised title, description and keywords fields. Alternatively, search terms may be qualified by prefixing them with a field name and a colon, e.g. title:awesome. Clients should indicate the desired localisation as described in Localisation.

Request

GET /api/v1/search?q=description:rubbish&fields=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[]

Request

GET /api/v1/search?q=description:awesome HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

Package Details

/api/v1/package

Requests details for a specific package. Cleans the query string to ensure the user cannot be tricked into installing the wrong package, and enforces a single item in the response.

Request

GET /api/v1/package/org.example.awesomewidget?fields=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: xyz987

{
  "name": "org.example.awesomewidget",
  "title": "Awesome Widget",
  "description": "This is an awesome widget.",
  "price": 1.99,
  "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
}

Request

GET /api/v1/package/org.example.awesomewidget?fields=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 304 Not Modified
Vary: Accept-Language
ETag: xyz987
Date: Thu, 04 Jul 2013 18:44:24 BST

Request

GET /api/v1/package/org.example.nonexistent?fields=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 404 Not Found
Date: Thu, 04 Jul 2013 18:44:24 BST

Departments

/api/v1/departments

Returns a list of the departments currently used by the packages in the index. Appending department names to the URL allows for drill-down searching.

Request

GET /api/v1/departments HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games",
    "name": "Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
    "name": "Games/Board Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics",
    "name": "Graphics"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics/Drawing",
    "name": "Graphics/Drawing"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet",
    "name": "Internet"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Chat",
    "name": "Internet/Chat"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Mail",
    "name": "Internet/Mail"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Web+Browsers",
    "name": "Internet/Web Browsers"
  }
]

Request

GET /api/v1/departments/Games HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

{
  "name": "Games",
  "subdepartments": [
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Role-Playing",
      "name": "Games/Role-Playing"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
      "name": "Games/Board Games"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Puzzles",
      "name": "Games/Puzzles"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Card+Games",
      "name": "Games/Card Games"
    }
  ]
}

Highlights

/api/v1/highlights

Return a list of ranked "highlighted" packages for the given highlight. An empty search (i.e. /api/v1/highlights) will return results from a default highlight¹. The result is in the same format as a standard search.

Request

GET /api/v1/highlight/top-apps HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
X-Ubuntu-Frameworks: ubuntu-sdk-13.10,ubuntu-sdk-14.04-dev
X-Ubuntu-Architecture: armhf

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

We don't store anything user-identifiable so "you might be interested in"-type recommendations are not going to be feasible without the client sending a list of what's installed, which is a privacy nightmare. (1)

Ubuntu Wiki