NAV Navbar
http python javascript ruby
  • Introduction
  • Authentication
  • JSON-API conventions
  • Projects
  • SubjectSetImports
  • Users
  • Introduction

    Panoptes API is a crowdsourcing system based on serving subjects, images or other media, to volunteers and asking them to classify the subject based on a defined workflow for the subject type.

    Authentication

    Panoptes API provides an OAuth 2.0 authentication mechanism. (TODO: Describe Authentication Workflow).

    How do I authenticate with the Zooniverse API?

    Some requests to the Zooniverse API will respond without supplied authentication details as they are public but most require an authentication token to be supplied to access our systems as a known user.

    This authentication token is known as a bearer token and is usually supplied as a HTTP Authorization header with the value prefixed by Bearer and then the token data. For example:

    Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

    The bearer tokens we produce are actually JWT tokens in and of themselves. This means that if you have a bearer token, you can unpack it, verify that it was signed by our servers, and find your user ID, your permissions and some other things.

    Tools to interact with the API

    You can use Postman, Paw, cURL or one of our official Zooniverse libraries to authenticate and create requests to the API:

    Example using Postman

    Postman is a graphical tool to help you make API calls.

    How do you get a token?

    POST /oauth/token HTTP/1.1
    Content-Type: application/json
    
    {
      "grant_type": "client_credentials",
      "client_id": "YOUR_CLIENT_ID",
      "client_secret": "YOUR_CLIENT_SECRET"
    }
    
    from panoptes_client import Panoptes
    client = Panoptes.connect(username='', password='')
    print(client.get_bearer_token())
    
    require 'rubygems'
    require 'panoptes-client'
    client = Panoptes::Client.new(auth: {client_id: CLIENT_ID, client_secret: CLIENT_SECRET})
    response = client.panoptes.connection.get('/me')
    auth = response.env.request_headers["Authorization"]
    token = auth.gsub(/Bearer (.*)/, "\\1")
    puts token
    

    There are multiple ways to get an OAuth bearer token from our authentication system, we support most of the OAuth grant types to convert credentials to a token. However our system restricts non-vetted apps from using particular flows, e.g. password (mostly to stop phishing sites harvesting our user credentials).

    The easiest to get started with is to use OAuth and the client credentials OAuth flow. To do this you will need to create an OAuth application with our system.

    Complete the form like the image below: OAuth Application

    Tick all the resource scopes you'll be needing access to (start with all). Once setup note your client ID and secret for that app and that this app will be linked to your user account. It is imperative that you do not share the OAuth application secret as it can gain access to your Zooniverse account as if you were using the system.

    There are other OAuth flows that can be used to get bearer tokens, please email contact@zooniverse.org to find out more.

    Setup postman to use the client credentials to get API bearer tokens

    I won’t go into details on how to setup postman, there are lots of docs that help with this.

    You will need to setup a request to https://signin.zooniverse.org/api/me and under the authorization tab, select OAuth 2.0, and click the ‘Get New Access Token’ button. Complete the forms like the images below:

    initial postman req

    get new access token

    1. Token name is used to identify the token in Postman, this is for your reference only
    2. Select the grant type to be ‘Client Credentials’
    3. Use the client ID from the OAuth application you setup at signin.zooniverse.org
    4. Same for client secret
    5. Scope is a lowercase, space separated list of all OAuth application scopes you selected at signin.zooniverse.org, in the image above for my testing app i only ticked user scope.
      • All scopes - user project group collection classification subject medium organization translation public
    6. Client authentication - send credentials in the request body
    7. Press ‘Request Token’ button
      • If you can’t get a token and get a 422 response from the API ensure your scopes match exactly the list you selected for the app at signin.zooniverse.org earlier.
      • Use the Postman console to debug the requests
    8. If successful you’ll see a screen with an access token, scroll down and press the ‘Use Token’ button

    access token

    Now the request will have the authentication token setup for use in postman.

    req with access token

    Add extra Zooniverse API headers for each request

    The Zooniverse API requires some extra headers to access the API endpoints, specifically setting the Accept and Content-Type headers.

    Click on the ‘Headers’ tab and set the following header key value pairs as per the image below.

    1. Accept: application/vnd.api+json; version=1
    2. Content-Type: application/json

    add extra req headers

    Without the above extra headers any response to API endpoints will return a HTTP 404 response status code.

    Ready to send requests to the Zooniverse API

    The request is now ready to run, click the ‘Send’ button to send the request to the Zooniverse API, you should see the response show up.

    This response is to the https://signin.zooniverse.org/api/me endpoint and requests the information about the owner of the bearer token, i.e. ‘me’.

    api me response

    Debugging requests and responses

    Again any request failures can be debugged using the postman console, here you get a detailed view each request / response that postman is sending.

    postman console

    Some common failure modes: 1. Can’t get a bearer token from OAuth application credentials + Ensure the app id and secret match exactly what is showing in https://signin.zoonivese.org/oauth/applications + Ensure the scopes match exactly, OAuth app has user selected, scopes should be ‘user public’ 0. Get a 404 on /api/* endpoints request + Ensure the extra Accept and content-type headers are set on the requests, without them the API won’t route requests properly.

    JSON-API conventions

    Resource(s) themselves

    {
      "users": [{
        "id": 123,
        ...
      }],
      ...
    }
    

    If you request a resource, you will find the results under a top-level key with the plural name of the resource. So for instance, if you a single specific user, you will find the user record under the users key.

    {
      "projects": [{
        ...,
        "links": {
          "workflows": [123],
          "avatar": {"href": "/projects/123/avatar", "type": "avatars"},
          ...
        }
      }],
      "links": {
        "projects.workflows": {
          "href": "/workflows?project_id={projects.id}",
          "type": "workflows"
        },
        "projects.avatar": {
          "href": "/projects/{projects.id}/avatar",
          "type": "media"
        },
        ...
      }
    }
    

    Any resource returned will specify a list of linked resources under its links attribute. Definition on where to request those linked resources can be found under the top-level links key (as opposed to the per-resource links).

    Pagination

    {
      "page": 1,
      "page_size": 2,
      "count": 28,
      "include": [],
      "page_count": 14,
      "previous_page": 14,
      "next_page": 2,
      "first_href": "/users?page_size=2",
      "previous_href": "/users?page=14page_size=2",
      "next_href": "/users?page=2&page_size=2",
      "last_href": "/users?page=14&page_size=2"
    }
    

    When requesting a list of resources, rather than a single resource, the response will include a top-level meta key. For performance reasons, results are returned in pages. You can use the data under the meta key to automatically navigate these paginated results.

    Including linked resources

    TODO

    Projects

    {
        "projects": [{
            "id": "1",
            "display_name": "Galaxy Zoo",
            "classifications_count": "1000000",
            "subjects_count": "10000",
            "created_at": "2014-03-24T10:42:21Z",
            "updated_at": "2014-03-24T10:42:21Z",
            "available_languages": ["en"],
            "title": "Galaxy Zoo",
            "description": "A Project ...",
            "guide": [{
                "image": "http://something.example.com/delorean.jpg",
                "explanation": "our ride"
            }],
            "team_members": [{
                "name": "Doc Brown",
                "bio": "",
                "twitter": "thatsmydelorean",
                "insitution": nil
            }],
            "science_case": "88mph + 1.21 GW = 1955",
            "introduction": "asdfasdf",
            "background_image": "http://test.host/12312asd.jp2",
            "private": false,
            "faq": "This project uses..",
            "result": "We found amazing things",
            "education_content": "Educator content goes here",
            "retired_subjects_count": "5000",
            "configuration": { "option_1": "value" },
            "beta": "false",
            "approved": "true",
            "live": "true",
            "links": {
                "owner": {
                    "id": "2",
                    "display_name": "Owner 2",
                    "href": "/users/2",
                    "type": "user"
                }
            }
        }],
        "links": {
            "projects.subjects": {
                "href": "/subjects?project_id={projects.id}",
                "type": "subjects"
            },
            "projects.classifications": {
                "href": "/classifications?project_id={projects.id}",
                "type": "classifications"
            },
            "projects.workflows": {
                "href": "/workflows?project_id={projects.id}",
                "type": "workflows"
            },
            "projects.subject_sets": {
                "href": "/subject_sets/{projects.subject_sets}",
                "type": "subject_sets"
            },
            "projects.owner": {
                "href": "/{projects.owner.href}",
                "type": "owners"
            }
        }
    }
    

    A Project is a collection of subjects and task workflows that a volunteer performs on subjects. The project also holds supplementary text describing the tasks volunteers perform.

    It has the following attributes:

    Attribute Type Description
    id integer read-only
    display_name string
    classifications_count integer read-only
    subjects_count integer read-only
    created_at string read-only
    updated_at string read-only
    available_languages array(string)
    title string
    description string
    guide array(hash)
    team_members array(hash)
    science_case string
    introduction string
    avatar ??
    background_image string
    private boolean
    faq string
    result string
    education_content string
    retired_subjects_count integer read-only
    configuration hash
    beta boolean
    approved boolean
    live boolean

    id, created_at, updated_at, user_count, and classifications_count are set by the API.

    List All Projects

    GET /api/projects HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    

    Retrieve a single Project

    GET /api/projects/123 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    

    Create a Project

    POST /api/projects HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    
    {
        "projects": {
            "name": "galaxy_zoo",
            "description": "A doubleplus good project",
            "primary_language": "en-us",
            "private": true,
            "links": {
                "workflows": ["1", "2"]
                "owner": {
                    "id": 10,
                    "display_name": "Owner 2",
                    "type": "user_groups",
                    "href": "user_groups/10"
                }
            }
        }
    }
    

    Requires at least a display_name, description, primary_language and private. Workflows and SubjectSets added as links will be copied and their new ids returned.

    Edit a single Project

    PUT /api/projects/123 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    
    {
        "projects": {
            "display_name": "Galaxy Zooooooooo!",
            "links": {
                "workflows": ["1"],
                "subject_sets": ["10"]
            }
        }
    }
    

    A User must be the owner of a project or have update permissions to edit the resource. Workflow and Subject Set links may be edited. Removing a subject set or workflow causes the subject set or workflow to be destroyed. Adding a workflow or subject set causes the original to be copied and a new id to be returned.

    Setting has may links through a PUT, while supported, is not recommended. Instead, use the link endpoints explained below.

    Destroy a single Project

    DELETE /api/projects/123 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    

    A user may destroy a project they own or have destroy permissions for.

    Response will be an HTTP 204.

    SubjectSetImports

    {
        "source_url": "https://path.to/some/manifest.json",
        "links": {
            "subject_set": "1"
        }
    }
    

    This resource is useful when a researcher has a large batch to add. The upload process goes as follows:

    1. A researcher uploads image files into an S3 bucket, or other publicly accessible location. She also creates a manifest file (format described below). The manifest file describes "subjects", which is the Zooniverse term for something that is to be classified. A subject is made up of a unique UUID, one or more URLs that host media files, and metadata described as key-value pairs.
    2. In the Zooniverse project builder, the researcher navigates to the subject set to which they would like to have the subjects added. The researcher then clicks "Import manifest", and enters the URL to the manifest (as given to them by step 2)

    This calls a Zooniverse API which enqueues a background job on the Zooniverse servers. In the background, the Zooniverse's job workers download the manifest file, and process it, creating subjects in the specified subject set. Additional features beyond the MVP: When the science platform opens a new browser tab/window to send the researcher to the project builder, it can pass along the manifest URL for the small sample of <100 subjects as URL query parameters. The Zooniverse project builder will then make the Import button more prominent and prefill the URL. After approval, when the researcher needs to get a full set of data into their Zooniverse project, at the end the supertask can call the Zooniverse API to trigger the import rather than requesting the researcher to go back to the project builder and click a button.

    Creating a new import

    POST /api/subject_set_imports HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    
    {
        "subject_set_imports": {
           "source_url": "https://path.to/some/manifest.json",
           "links": {
               "subject_set": "1"
           }
        }
    }
    

    Returns a SubjectSetImport resource with an ID. This resource can be polled in order to get the status of the import.

    Retrieve a single import

    GET /api/subject_set_imports/1 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    

    Manifest CSV format

    uuid,location:1,location:2,metadata:size,metadata:cuteness
    1,https://placekitten.com/200/300.jpg,https://placekitten.com/200/100.jpg,small,cute
    2,https://placekitten.com/400/900.jpg,https://placekitten.com/500/100.jpg,large,cute
    

    The manifest.csv file is contains one row per subject. See the example to the right. It is expected to have the following columns:

    Users

    {
        "users": [{
            "id": 1,
            "created_at": "2014-03-20T00:15:47Z",
            "updated_at": "2013-09-30T10:20:32Z",
            "credited_name": "Dr. Stuart Lynn",
            "login": "stuart",
            "display_name": "Stuart",
            "owner_name": "stuart",
            "email": "stuart@zoooniverse.org",
            "zooniverse_id": "123432",
            "classifications_count": "104",
            "languages": ["en-gb", "es-mx"]
        }],
        "links": {
            "users.projects": {
                "href": "/projects?owner={users.owner_name}",
                "type": "projects"
            },
            "users.user_groups": {
                "href": "/user_groups?user_id={users.id}",
                "type": "user_groups"
            },
            "users.subjects": {
                "href": "/subjects?user_id={users.id}",
                "type": "subjects"
            },
            "users.collections": {
                "href": "/collections?owner={users.owner_name}",
                "type": "collections"
            }
        }
    }
    

    A User is representation of the identity and contributions of a volunteer.

    Attribute Type Description
    id Integer
    created_at string
    updated_at string
    credited_name string Publicly available name with which a volunteer will be credited on papers, posters, etc
    login string
    display_name string
    email string The email of the user
    zooniverse_id string
    classifications_count integer Number of classifications this user has made site-wide
    languages Array(String) Array of language identifier string, in order from most to least preferred. Used to determine which language to show translated projects in.

    id, zooniverse_id, created_at, updated_at, and classification_count are created and updated by the Panoptes API.

    credited_name is the publicly available name with which a volunteer will be credited on papers, posters, etc. When serialized, if an @ character is found, the user's login will be returned instead for privacy reasons.

    The email attribute is only available if the requesting user is an administrator or the user resource being requested is that of the requesting user.

    List Users

    GET /api/users HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    

    Parameters:

    Retrieve a single user

    GET /api/users/1 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    
    Parameter Type Default Description
    id integer ID of the User as an integer key

    Edit a single User

    PUT /api/users/123 HTTP/1.1
    Accept: application/vnd.api+json; version=1
    Content-Type: application/json
    
    {
        "users": {
            "credited_name": "Dr. Stuart Lynn, DDS"
        }
    }
    

    The currently logged in User may edit their record by sending a partial representation of the resource including their changes. A User cannot edit linked resources.

    Destroy a single User

    DELETE /users/123 HTTP/1.1
    

    The current logged in User may delete themself. This does not fully remove the user resource; instead, it deactivates their projects and removes personally identifying information including the credited_name and email address.