{"id":2476,"date":"2016-07-19T12:42:34","date_gmt":"2016-07-19T03:42:34","guid":{"rendered":"http:\/\/avinton.com\/?page_id=2476"},"modified":"2021-11-09T16:12:41","modified_gmt":"2021-11-09T07:12:41","slug":"postgresql-data-analyses","status":"publish","type":"page","link":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/","title":{"rendered":"PostgreSQL Data Analyses"},"content":{"rendered":"<div class=\"wpb-content-wrapper\"><p>[vc_row][vc_column][vc_column_text]<\/p>\n<h3>PostgreSQL Data Analyses<\/h3>\n<h4>Load data into PostgreSQL<\/h4>\n<pre class=\"\">cd \/var\/tmp\/\r\nwget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv\r\nsudo su\r\nsu postgres\r\npsql -d avinton\r\ncreate table japan_cities (prefecture text, city text, ward text, population integer);\r\ncopy japan_cities from '\/var\/tmp\/japan_cities.tsv' NULL 'NULL';\r\n\\q<\/pre>\n<h4>Data Cleaning<\/h4>\n<p>Connect to PostgreSQL DB using PGAdmin and run the following to clean the data<\/p>\n<pre class=\"\">create table japan_cities2 as (\r\nselect a.*, row_number() over() as id from japan_cities as a\r\n);\r\n\r\ndrop table japan_cities;\r\ncreate table japan_cities as (\r\nselect\r\n    case when prefecture is null then (select c.prefecture from japan_cities2 as c where c.id = (select max(b.id) from japan_cities2 b where b.id &lt; a.id and b.prefecture is not null)) end as prefecture,\r\n    (select c.city from japan_cities2 as c where c.id = (select max(b.id) from japan_cities2 b where b.id &lt;= a.id and b.city is not null)) as city,\r\n    ward, population\r\nfrom japan_cities2 as a);\r\n\r\ndrop table japan_cities2;\r\n\r\ndelete from japan_cities where population is null;\r\nalter table japan_cities add column city_ward text;\r\nupdate japan_cities set city_ward = 'c' where ward is null;\r\nupdate japan_cities set city_ward = 'w' where ward is not null;\r\n\r\nselect * from japan_cities;<\/pre>\n<h4>Exercises<\/h4>\n<p>Have a look at the data first and try to understand its structure. Then try these exercises below:<\/p>\n<p>&nbsp;<\/p>\n<p>1. Which is the ward with the highest population? (hint: try using order by and limit clause <a href=\"https:\/\/www.postgresql.org\/docs\/9.2\/static\/queries-limit.html\" target=\"_blank\" rel=\"noopener noreferrer\">Limit<\/a>)<\/p>\n<p>2. What is the standard deviation for the city populations?<\/p>\n<p>3. How many cities are there in Hokkaido?<\/p>\n<p>4. How many wards are there in total in Japan?<\/p>\n<p>5. list the prefectures and their corresponding population<\/p>\n<p>6. list the prefectures only in order of descending population<\/p>\n<p>7. list the prefectures and the corresponding city within it with the highest population.<\/p>\n<p>Challenge:<br \/>\nTry to use PostgreSQL&#8217;s windowing functions to generate the following:<br \/>\nFor each city show the prefecture, city, smallest_ward, largest_ward, total_city_population.<\/p>\n<p>The table should contain 1 line per city and be ordered by city population in descending order.[\/vc_column_text][\/vc_column][\/vc_row]<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>[vc_row][vc_column][vc_column_text] PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres psql -d avinton create table japan_cities (prefecture text, city text, ward text, population integer); copy japan_cities from &#8216;\/var\/tmp\/japan_cities.tsv&#8217; NULL &#8216;NULL&#8217;; \\q Data Cleaning Connect to PostgreSQL DB using PGAdmin and run the following to clean the data create table<br \/><a href=\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\" class=\"more\">Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":1906,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-2476","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>PostgreSQL Data Analyses - Avinton Japan<\/title>\n<meta name=\"description\" content=\"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PostgreSQL Data Analyses - Avinton Japan\" \/>\n<meta property=\"og:description\" content=\"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres\" \/>\n<meta property=\"og:url\" content=\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\" \/>\n<meta property=\"og:site_name\" content=\"Avinton Japan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Avintons\/\" \/>\n<meta property=\"article:modified_time\" content=\"2021-11-09T07:12:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/avinton.com\/wp-content\/uploads\/2020\/02\/avinton-japan-big-data.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@AvintonJapan\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\",\"url\":\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\",\"name\":\"PostgreSQL Data Analyses - Avinton Japan\",\"isPartOf\":{\"@id\":\"https:\/\/avinton.com\/en\/#website\"},\"datePublished\":\"2016-07-19T03:42:34+00:00\",\"dateModified\":\"2021-11-09T07:12:41+00:00\",\"description\":\"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres\",\"breadcrumb\":{\"@id\":\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/avinton.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Avinton Academy\",\"item\":\"https:\/\/avinton.com\/en\/academy\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"PostgreSQL Data Analyses\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/avinton.com\/en\/#website\",\"url\":\"https:\/\/avinton.com\/en\/\",\"name\":\"Avinton Japan\",\"description\":\"Tailored Solutions and Consulting in AI and Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/avinton.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"PostgreSQL Data Analyses - Avinton Japan","description":"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/","og_locale":"en_US","og_type":"article","og_title":"PostgreSQL Data Analyses - Avinton Japan","og_description":"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres","og_url":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/","og_site_name":"Avinton Japan","article_publisher":"https:\/\/www.facebook.com\/Avintons\/","article_modified_time":"2021-11-09T07:12:41+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/avinton.com\/wp-content\/uploads\/2020\/02\/avinton-japan-big-data.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@AvintonJapan","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/","url":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/","name":"PostgreSQL Data Analyses - Avinton Japan","isPartOf":{"@id":"https:\/\/avinton.com\/en\/#website"},"datePublished":"2016-07-19T03:42:34+00:00","dateModified":"2021-11-09T07:12:41+00:00","description":"PostgreSQL Data Analyses Load data into PostgreSQL cd \/var\/tmp\/ wget http:\/\/avinton.com\/wp-content\/uploads\/2016\/07\/japan_cities.tsv sudo su su postgres","breadcrumb":{"@id":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/avinton.com\/en\/academy\/postgresql-data-analyses\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/avinton.com\/en\/"},{"@type":"ListItem","position":2,"name":"Avinton Academy","item":"https:\/\/avinton.com\/en\/academy\/"},{"@type":"ListItem","position":3,"name":"PostgreSQL Data Analyses"}]},{"@type":"WebSite","@id":"https:\/\/avinton.com\/en\/#website","url":"https:\/\/avinton.com\/en\/","name":"Avinton Japan","description":"Tailored Solutions and Consulting in AI and Big Data","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/avinton.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/2476","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/comments?post=2476"}],"version-history":[{"count":13,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/2476\/revisions"}],"predecessor-version":[{"id":60568,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/2476\/revisions\/60568"}],"up":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1906"}],"wp:attachment":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/media?parent=2476"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/categories?post=2476"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/tags?post=2476"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}