ClickPackageIndex

Differences between revisions 28 and 29

Contents

Click Package Index

Click Package Index

Contact: James Tait (JamesTait)

Lucene-backed repository for Click Application metadata
Public front-end will serve as access control, apply sane defaults and massage request and response for ease of use.
Interfaces with:
- Software Centre Agent
  - App developer defines app in Software Centre website
  - App developer uploads packaged app
  - Software Centre Agent pushes metadata to Click Package Index
    - Metadata harvested from Click Manifest
    - Verified and corrected by developer
- Dash
  - "Surfacing", i.e. first view of app lens before querying
  - "Search", i.e. list of apps that match search criteria
  - "Detail", i.e. full metadata for a given app

Schema

Field Types

Name	Class	Notes
string	`solr.StrField`	Untokenised text.
boolean	`solr.BoolField`	True or False.
float	`solr.TrieFloatField`	Floating point number.
long	`solr.TrieLongField`	Long integer.
date	`solr.TrieDateField`	Date of the format `'YYYY-MM-DD"T"hh24:mm:ss[.nnn]"Z"'`, e.g. `'2013-06-27T12:39:45.776Z'`
url	`solr.StrField`	Untokenised text.
text_general	`solr.TextField`	Tokenised text. Reasonable cross-language defaults. `StandardTokenizer`, case-insensitive stop words, and down-casing. Query-time synonyms.
text_lang	`solr.TextField`	Localised tokenised text. `StandardTokenizer`, case-insensitive stop words, down-casing, protected words and minimal stemming. Query-time synonyms. See Localisation below.
key_value	`solr.TextField`	Key-value pair, i.e. `'key\|value'`.
category	`solr.TextField`	Tokenised text, with a semi-colon delimiter and down-casing.

Fields

Name	Type	Multi-value	Searchable	Retrievable	Required	Translatable	Notes
name	string	false	true	true	true	false	e.g. org.example.my_app
version	string	false	true	true	true	false
click_version	string	false	true	true	true	false
framework	string	true	true	true	true	false	Deprecated This field should not be used as a search parameter. The list of supported frameworks should be passed as a comma-separated list in the X-Ubuntu-Frameworks header instead. If both the header and the parameter are present, the header will take precedence.
license	text_general	false	true	true	true	false	License under which the software is made available.
download_url	url	false	false	true	true	false
title_lang	text_lang	false	true	true	true (en)	true	Display name, as presented in menus.
description_lang	text_lang	false	true	true	true (en)	true	Full description of the app's functionality.
changelog	text_general	false	true	true	false	false	List of changes in this version.
text_lang	text_lang	true	true	false	false	true	Default searchable field containing tokens from title, description and keywords.
keywords_lang	text_lang	true	true	true	false	true
price	float	false	true	true	false	true (currencies)
support_url_lang	url	false	false	true	false	true	URL to localised support channel.
terms_of_service_lang	url	false	false	true	false	true	URL to localised ToS.
website_lang	url	false	false	true	false	true
binary_filesize	long	false	false	true	false	false	Size of the downloadable package, in bytes.
icon_url	url	false	false	true	false	false	URL to a 64x64px PNG image to use as an application icon, e.g. in menus.
icon_urls	string	true	true	true	false	false	List of delimited strings in the format ${size}\|${url} describing URLs to square PNG icons of various sizes.
screenshot_url	url	false	false	true	false	false	URL to a screen shot.
screenshot_urls	url	true	false	true	false	false	List of URLs to other screenshots.
company_name	text_general	false	true	true	false	false
developer_name	text_general	false	true	true	false	false	The full name of the developer profile which uploaded the package.
publisher	text_general	false	true	true	false	false	The company name or the full name of the developer profile which uploaded the package if the company name is not available.
date_published	date	false	true	true	false	false	Date of publication.
video_urls	url	true	false	true	false	false
license_key_path	string	false	false	true	false	false
requires_license_key	boolean	false	true	true	false	false
whitelist_country_codes	string	true	true	false	false	false
blacklist_country_codes	string	true	true	false	false	false
prices	string	true	true	true	false	false
architecture	string	true	true	true	false	false	Deprecated This field should not be used as a search parameter. The client architecture should be passed in the X-Ubuntu-Architecture header instead. If both the header and the parameter are present, the header will take precedence.
category	category	false	true	true	false	false	See the Software Centre Genre Spec and the freedesktop.org Desktop Menu Spec
department	string	true	true	true	false	false	See the Software Centre Genre Spec and the freedesktop.org Desktop Menu Spec
highlights	key_value	true	true	true	false	false

Localisation

Fields marked as translatable will have localised variants whose field name is suffixed with one of the following country codes:

Suffix	Language
ar	Arabic
bg	Bulgarian
ca	Catalan
cn	Chinese
cz	Czech
da	Danish
de	German
el	Greek
en	English
es	Spanish
eu	Basque
fa	Persian
fi	Finnish
fr	French
ga	Irish
gl	Galician
hi	Hindi
hu	Hungarian
hy	Armenian
id	Indonesian
it	Italian
ja	Japanese
kr	Korean
lv	Latvian
nl	Dutch
no	Norwegian
pt	Portuguese
ro	Romanian
ru	Russian
sv	Swedish
th	Thai
tr	Turkish

For queries:

If the query specifies a localisation (e.g. q=description_en:foo), we will use it.
Similarly, if a specific localised result field is requested (e.g. fl=description_en), we will use it.
In the general case, clients should use the non-localised field name in queries (q=description) and result fields (fl=description) and provide an Accept-Language header, which we will use to choose the localised variant.
If the request specifies an alternative localisation via the lang query parameter (e.g. lang=en) we will use that in preference to the language in the Accept-Language header.
If the request specifies a localisation which is unavailable, the English localisation will be returned.

This keeps the query API flexible, but avoids unnecessary complexity in the client for the general case. We'll need to be wary of caching, in particular - Vary: Accept-Language and ETag headers will be required at least.

API

Guiding principles:

JSON-based
Discoverable
Browseable

API Root

/api/v1

Provides clickable links to the other endpoints for discoverability.

Request

GET /api/v1 HTTP/1.1
Host: search.apps.ubuntu.com

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
    "search": "http://search.apps.ubuntu.com/api/v1/search",
    "package": "http://search.apps.ubuntu.com/api/v1/package",
    "highlights": "http://search.apps.ubuntu.com/api/v1/highlights",
    "departments": "http://search.apps.ubuntu.com/api/v1/departments"
}

Search

/api/v1/search

Proxies requests to Solr's SearchHandler. A subset of the standard Solr syntax is used for querying. Clients should indicate the desired localisation as described in Localisation.

Request

GET /api/v1/search?q=description:rubbish&fl=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 204 No Content
Vary: Accept-Language

Request

GET /api/v1/search?q=framework:ubuntu-sdk-13.10,description:awesome HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

Package Details

/api/v1/package

Requests details for a specific package. Cleans the query string to ensure the user cannot be tricked into installing the wrong package, and enforces a single item in the response.

Request

GET /api/v1/package/org.example.awesomewidget?fl=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: xyz987

{
  "name": "org.example.awesomewidget",
  "title": "Awesome Widget",
  "description": "This is an awesome widget.",
  "price": 1.99,
  "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
}

Request

GET /api/v1/package/org.example.awesomewidget?fl=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987

Response

HTTP/1.1 304 Not Modified
Vary: Accept-Language
ETag: xyz987
Date: Thu, 04 Jul 2013 18:44:24 BST

Request

GET /api/v1/package/org.example.nonexistent?fl=name,title,description,price,icon_url HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en

Response

HTTP/1.1 404 Not Found
Date: Thu, 04 Jul 2013 18:44:24 BST

Departments

/api/v1/departments

Returns a list of the departments currently used by the packages in the index. Appending department names to the URL allows for drill-down searching.

Request

GET /api/v1/departments HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games",
    "name": "Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
    "name": "Games/Board Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics",
    "name": "Graphics"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics/Drawing",
    "name": "Graphics/Drawing"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet",
    "name": "Internet"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Chat",
    "name": "Internet/Chat"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Mail",
    "name": "Internet/Mail"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Web+Browsers",
    "name": "Internet/Web Browsers"
  }
]

Request

GET /api/v1/departments/Games HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

{
  "name": "Games",
  "subdepartments": [
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Role-Playing",
      "name": "Games/Role-Playing"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
      "name": "Games/Board Games"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Puzzles",
      "name": "Games/Puzzles"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Card+Games",
      "name": "Games/Card Games"
    }
  ]
}

Highlights

/api/v1/highlights

Return a list of ranked "highlighted" packages for the given highlight. An empty search (i.e. /api/v1/highlights) will return results from a default highlight <<FootNote(We don't store anything user-identifiable, so "you might be interested in"-type recommendations are not going to be feasible without the client sending a list of what's installed, which is a privacy nightmare.)>>. The result is in the same format as a standard search.

Request

GET /api/v1/highlight/top-apps HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]

AppStore/Interfaces/ClickPackageIndex (last edited 2018-04-01 09:09:46 by popey)

-  ⇤ ← Revision 28 as of 2014-03-13 16:41:59 → 
  Size: 14926
  Editor: dhcp-host
  Comment:
+   ← Revision 29 as of 2014-03-13 17:27:43 → ⇥
  Size: 18976
  Editor: dhcp-host
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 6:
- * Solr-backed repository for Click Application metadata
+ * Lucene-backed repository for Click Application metadata
 Line 13:
-    * First iteration, metadata is entered by developer
    * Later to be harvested from Click Manifest
+    * Metadata harvested from Click Manifest
    * Verified and corrected by developer
 Line 21:
-== Solr Schema ==
+== Schema ==
 Line 43:
-|| framework                || string       || true              || true             || true              || true           || false              ||             ||
+|| framework                || string       || true              || true             || true              || true           || false              || '''Deprecated'''  This field should not be used as a search parameter.  The list of supported frameworks should be passed as a comma-separated list in the X-Ubuntu-Frameworks header instead.  If both the header and the parameter are present, the header will take precedence. ||
 Line 70:
-|| architecture             || string       || true              || true             || true              || false          || false              ||             ||
+|| architecture             || string       || true              || true             || true              || false          || false              || '''Deprecated'''  This field should not be used as a search parameter.  The client architecture should be passed in the X-Ubuntu-Architecture header instead.  If both the header and the parameter are present, the header will take precedence. ||
 Line 147:
-    "package": "http://search.apps.ubuntu.com/api/v1/package"
+    "package": "http://search.apps.ubuntu.com/api/v1/package",
    "highlights": "http://search.apps.ubuntu.com/api/v1/highlights",
    "departments": "http://search.apps.ubuntu.com/api/v1/departments"
-Line 154:
+Line 156:
-Proxies requests to Solr's `SearchHandler`.  A subset of the standard Solr syntax is used for querying.  Clients should indicate the desired localisation as described in [[#Localisation|Localisation]].  An empty search (i.e. `/api/v1/search` or `/api/v1/search?q=`) will return "recommended" results<<FootNote(We don't store anything user-identifiable, so "you might be interested in"-type recommendations are not going to be feasible without the client sending a list of what's installed, which is a privacy nightmare.  For now we can flag some apps as "featured" and return those.)>>.
+Proxies requests to Solr's `SearchHandler`.  A subset of the standard Solr syntax is used for querying.  Clients should indicate the desired localisation as described in [[#Localisation|Localisation]].
-Line 187:
+Line 189:
-    "description": "This is an awesome launcher.",
+    "publisher": "Awesome Widget Company",
-Line 195:
+Line 197:
-    "description": "This is an awesome widget.",
+    "publisher": "Awesome Widget Company",
-Line 203:
+Line 205:
-=== Details ===
+=== Package Details ===
-Line 261:
+Line 263:
+=== Departments ===
`/api/v1/departments`

Returns a list of the departments currently used by the packages in the index.
Appending department names to the URL allows for drill-down searching.

==== Request ====
{{{
GET /api/v1/departments HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games",
    "name": "Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
    "name": "Games/Board Games"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics",
    "name": "Graphics"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Graphics/Drawing",
    "name": "Graphics/Drawing"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet",
    "name": "Internet"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Chat",
    "name": "Internet/Chat"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Mail",
    "name": "Internet/Mail"
  },
  {
    "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Web+Browsers",
    "name": "Internet/Web Browsers"
  }
]
}}}

==== Request ====
{{{
GET /api/v1/departments/Games HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

{
  "name": "Games",
  "subdepartments": [
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Role-Playing",
      "name": "Games/Role-Playing"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games",
      "name": "Games/Board Games"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Puzzles",
      "name": "Games/Puzzles"
    },
    {
      "resource_url": "https://search.apps.ubuntu.com/api/v1/departments/Games/Card+Games",
      "name": "Games/Card Games"
    }
  ]
}
}}}

=== Highlights ===
`/api/v1/highlights`

Return a list of ranked "highlighted" packages for the given highlight.  An
empty search (i.e. `/api/v1/highlights`) will return results from a default
highlight <<FootNote(We don't store anything user-identifiable, so "you might be
interested in"-type recommendations are not going to be feasible without the
client sending a list of what's installed, which is a privacy nightmare.)>>.
The result is in the same format as a standard search.

==== Request ====
{{{
GET /api/v1/highlight/top-apps HTTP/1.1
Host: search.apps.ubuntu.com
Accept-Language: en
If-None-Match: xyz987
}}}

==== Response ====
{{{
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Language
ETag: abc123

[
  {
    "name": "org.example.awesomelauncher",
    "title": "Awesome Launcher",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomelauncher/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomelauncher"
  },
  {
    "name": "org.example.awesomewidget",
    "title": "Awesome Widget",
    "publisher": "Awesome Widget Company",
    "price": 1.99,
    "icon_url": "http://example.org/media/awesomewidget/icons/icon16.png",
    "resource_url": "http://search.apps.ubuntu.com/api/v1/package/org.example.awesomewidget"
  }
]
}}}

Ubuntu Wiki