Module 8

Projects

Creating a project

Type the following in your terminal:

$ lein new clojurebridge global-growth

Leiningen

manages Clojure projects
tool you run in your terminal
silly name

Project structure

.gitignore
doc/intro.md
project.clj
resources/
README.md
src/global_growth/core.clj
test/global_growth/core_test.clj

project.clj

configuration file for Leiningen
lists dependencies
contains information about program author, description and more

src/global_growth/core.clj

This is where we will be writing much of our code.

Clojure programs can be made up of multiple files, but we are going to use just this one for now.

Try it out

Go to the command line and enter:

$ cd global-growth
$ lein run

Modifying a project

What happens when I run my program?

Open src/global_growth/core.clj.

What is in the -main function?

The -main function

Just an ordinary function with an odd name
The function called first when you run your program
Can call other functions

Using other functions from -main

(defn quotify
  [quote author]
  (str quote "\n\n-- " author))

(defn -main
  [& args]
  (println
   (quotify (str "A man who carries a cat by the tail learns "
                 "something he can learn in no other way.")
            "Mark Twain")))

Namespaces and organization

Namespaces let you organize your code into logical sections.

;; in src/global_growth/core.clj
(ns global-growth.core)

Dependencies

Dependencies are code libraries others have written you can reuse in your project.

Open project.clj and look for the :dependencies key.

:dependencies [[org.clojure/clojure "1.5.1"]
               [clj-http "0.9.0"]
               [cheshire "5.3.1"]]

Requiring dependencies

;; in src/global_growth/core.clj
(ns global-growth.core
  (:require [clj-http.client :as client]
            [cheshire.core :as json]))

Your first real program

The World Bank API

collection of world development indicators data
provided as JSON documents

What is an API?

Application Programming Interface
web pages for computers
one popular format: JSON

API in action

http://api.worldbank.org/countries/all/indicators/EN.POP.DNST?format=json&date=2010

Sample API response

[
  {
    "page": 1,
    "pages": 6,
    "per_page": "50",
    "total": 252
  },
  [
    {
      "indicator": {
        "id": "EN.POP.DNST",
        "value": "Population density (people per sq. km of land area)"
      },
      "country": {
        "id": "1A",
        "value": "Arab World"
      },
      "value": "25.5287276250072",
      "decimal": "0",
      "date": "2010"
    },
    {
      "indicator": {
        "id": "EN.POP.DNST",
        "value": "Population density (people per sq. km of land area)"
      },
      "country": {
        "id": "S3",
        "value": "Caribbean small states"
      },
      "value": "17.0236186241818",
      "decimal": "0",
      "date": "2010"
    }
  ]
]

Accessing APIs with Clojure

(client/get "http://api.worldbank.org/countries/all/indicators/EN.POP.DNST?format=json&date=2010")

;; elided and formatted
;;=> {:orig-content-encoding nil,
;;    :request-time 109, :status 200,
;;    :headers {"content-length" "10340",
;;              "content-type" "application/json;charset=utf-8"},
;;    :body "[{\"page\":1,\"pages\":6}]"}

Converting JSON

(json/parse-string "[{\"page\":1,\"pages\":6}]" true)
;;=> ({:page 1, :pages 6})

Exercise: Get the results from API call

Write a function called get-population-density that takes no arguments, and returns Clojure data from the World Bank API on population density.

You will need to make the web request, pull the :body value out of the response, and then parse the JSON.

(get-population-density)
;;=> ({:page 1, :pages 6, :per_page "50", :total 252}
;;    [{:indicator {:id "EN.POP.DNST", :value "Population density (people per sq. km of land area)"},
;;    :country {:id "1A", :value "Arab World"}, :value "25.5287276250072", :decimal "0", :date "2010"},
;;    ...])

Getting more information from the API

(defn get-api
  "Returns map representing API response."
  [path params]
  (let [url (str "http://api.worldbank.org" path)
        query-params (merge params {:format "json" :per_page 10000})
        response (json/parse-string
                  (:body
                   (client/get url {:query-params query-params})) true)
        metadata (first response)
        results (second response)]
    {:metadata metadata
     :results results}))

get-api

(get-api "/countries/all/indicators/EN.POP.DNST" {:date 2010})
;;=> {:metadata {:page 1, :pages 1, :per_page "10000", :total 252},
;;    :results [{:indicator {:id "EN.POP.DNST",
;;    :value "Population density (people per sq. km of land area)"},
;;    :country {:id "1A", :value "Arab World"}, :value "25.5287276250072",
;;    :decimal "0", :date "2010"} ...]}

Exercise: extracting the data we want

Write a function get-country-and-value that can take the return value of get-api and get the country names and values out of that value. get-country-and-value should return a vector of vectors.

(get-country-and-value
  (get-api "/countries/all/indicators/EN.POP.DNST" {:date 2010}))
;;=> [["Arab World" "25.5287276250072"]
;;    ["Caribbean small states" "17.0236186241818"]
;;    ...]

Removing unwanted data

Uncomment the definitions for remove-aggregrate-countries, countries, and get-indicator-values.

get-indicator-values

(defn get-indicator-values
  "Returns indicator values for a specified year for all countries."
  [indicator-code year]
  (let [response (get-api (str "/countries/all/indicators/"
                               indicator-code)
                          {:date (str year)})
        values (get-country-and-value response)]
    (for [[country value] values
          :when (and (not (nil? value))
                     (contains? @countries country))]
      [country (read-string value)])))

Get a list of countries and population density

(get-indicator-values "EN.POP.DNST" 2010)

What do you get?

Revisiting -main

Updating -main

Let's make our program print out a list of all countries and their population density.
Make sure -main is the last function in your file.

doseq

In order to print out all the countries and population densities, you will need a new statement, doseq.

doseq works like for but executes its body and returns nothing.

(doseq [name ["Akeelah" "Bhamini" "Cierra"]]
  (println name))

Exercise: Finish -main

Using doseq and println, write a -main function that prints out all the countries and their population densities from the World Bank API.

Use (get-indicator-values "EN.POP.DNST" 2010) to get the values you need.

Bonus exercise: Only show the top 10 countries

Change your -main function to only print out the top 10 countries and their population densities. You will need the sort-by function to make this work.