{"id":1931,"date":"2016-06-24T18:22:19","date_gmt":"2016-06-24T09:22:19","guid":{"rendered":"http:\/\/avinton.com\/?page_id=1931"},"modified":"2024-10-03T10:46:28","modified_gmt":"2024-10-03T01:46:28","slug":"loading-data-postgresql-redis","status":"publish","type":"page","link":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/","title":{"rendered":"Loading Data into PostgreSQL &#038; Redis"},"content":{"rendered":"<div class=\"wpb-content-wrapper\"><p>[vc_row][vc_column][vc_column_text]Open an SQL Editor within PGAdmin:<\/p>\n<p>Let&#8217;s create some test data<br \/>\nThe special generate_series command in PostgreSQL allows us to do this easily.<\/p>\n<pre class=\"\">create table test_data as(\r\nselect a as k, 'row' || a as val from (\r\nselect generate_series(1,100,1) as a\r\n) as x);<\/pre>\n<p>We have just loaded some data into a new table called test_data.<br \/>\nHave a look at the data using:<\/p>\n<pre class=\"\">select * from test_data;<\/pre>\n<p>Let&#8217;s add the redis command text column so we can generate the redis data too from the same data.<\/p>\n<pre class=\"\">alter table test_data add column rtext text;<\/pre>\n<p>Update the new column with the data from the same table adding in the REDIS Command syntax<\/p>\n<pre class=\"\">update test_data set rtext = 'SET ' || k || ' ' || val;<\/pre>\n<p>Let&#8217;s output the data from PostgreSQL table into a file called a.txt for sending to REDIS<br \/>\nOn the ubuntu VM Terminal:<\/p>\n<pre class=\"\">su postgres\r\ncd \/var\/tmp\/\r\npsql -d avinton -c 'select rtext from test_data;' -o a.txt<\/pre>\n<p>Now let&#8217;s change permissions for this new file so that all users can access<\/p>\n<pre class=\"\">chmod 777 a.txt<\/pre>\n<p>Let&#8217;s copi this file to a common area where users can access it<\/p>\n<pre class=\"\">cp a.txt \/var\/tmp\/<\/pre>\n<p>Lets use this file to load data to REDIS<br \/>\nExit from postgres user<\/p>\n<pre class=\"\">exit<\/pre>\n<p>Exit from root user<\/p>\n<pre class=\"\">exit<\/pre>\n<p>As normal user lets run the redis-cli and pipe the file<\/p>\n<pre class=\"\">cat \/var\/tmp\/a.txt | redis-cli --pipe<\/pre>\n<p>Now we should have the same data in both REDIS &amp; PostgreSQL DB (only 100 rows)<br \/>\nHave a look at the data using the REDIS Desktop as <a href=\"https:\/\/avinton.com\/en\/academy\/redis\/\" target=\"_blank\" rel=\"noopener\">here<\/a>[\/vc_column_text][\/vc_column][\/vc_row]<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>[vc_row][vc_column][vc_column_text]Open an SQL Editor within PGAdmin: Let&#8217;s create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table test_data as( select a as k, &#8216;row&#8217; || a as val from ( select generate_series(1,100,1) as a ) as x); We have just loaded some data into a new table<br \/><a href=\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\" class=\"more\">Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":1906,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-1931","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Loading Data into PostgreSQL &amp; Redis - Avinton Academy<\/title>\n<meta name=\"description\" content=\"Open an SQL Editor within PGAdmin: Let&#039;s create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Loading Data into PostgreSQL &amp; Redis - Avinton Academy\" \/>\n<meta property=\"og:description\" content=\"Open an SQL Editor within PGAdmin: Let&#039;s create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table\" \/>\n<meta property=\"og:url\" content=\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\" \/>\n<meta property=\"og:site_name\" content=\"Avinton Japan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Avintons\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-03T01:46:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/avinton.com\/wp-content\/uploads\/2020\/02\/avinton-japan-big-data.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@AvintonJapan\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\",\"url\":\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\",\"name\":\"Loading Data into PostgreSQL & Redis - Avinton Academy\",\"isPartOf\":{\"@id\":\"https:\/\/avinton.com\/en\/#website\"},\"datePublished\":\"2016-06-24T09:22:19+00:00\",\"dateModified\":\"2024-10-03T01:46:28+00:00\",\"description\":\"Open an SQL Editor within PGAdmin: Let's create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table\",\"breadcrumb\":{\"@id\":\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/avinton.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Avinton Academy\",\"item\":\"https:\/\/avinton.com\/en\/academy\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Loading Data into PostgreSQL &#038; Redis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/avinton.com\/en\/#website\",\"url\":\"https:\/\/avinton.com\/en\/\",\"name\":\"Avinton Japan\",\"description\":\"Tailored Solutions and Consulting in AI and Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/avinton.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Loading Data into PostgreSQL & Redis - Avinton Academy","description":"Open an SQL Editor within PGAdmin: Let's create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/","og_locale":"en_US","og_type":"article","og_title":"Loading Data into PostgreSQL & Redis - Avinton Academy","og_description":"Open an SQL Editor within PGAdmin: Let's create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table","og_url":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/","og_site_name":"Avinton Japan","article_publisher":"https:\/\/www.facebook.com\/Avintons\/","article_modified_time":"2024-10-03T01:46:28+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/avinton.com\/wp-content\/uploads\/2020\/02\/avinton-japan-big-data.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@AvintonJapan","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/","url":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/","name":"Loading Data into PostgreSQL & Redis - Avinton Academy","isPartOf":{"@id":"https:\/\/avinton.com\/en\/#website"},"datePublished":"2016-06-24T09:22:19+00:00","dateModified":"2024-10-03T01:46:28+00:00","description":"Open an SQL Editor within PGAdmin: Let's create some test data The special generate_series command in PostgreSQL allows us to do this easily. create table","breadcrumb":{"@id":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/avinton.com\/en\/academy\/loading-data-postgresql-redis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/avinton.com\/en\/"},{"@type":"ListItem","position":2,"name":"Avinton Academy","item":"https:\/\/avinton.com\/en\/academy\/"},{"@type":"ListItem","position":3,"name":"Loading Data into PostgreSQL &#038; Redis"}]},{"@type":"WebSite","@id":"https:\/\/avinton.com\/en\/#website","url":"https:\/\/avinton.com\/en\/","name":"Avinton Japan","description":"Tailored Solutions and Consulting in AI and Big Data","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/avinton.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1931","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/comments?post=1931"}],"version-history":[{"count":11,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1931\/revisions"}],"predecessor-version":[{"id":113698,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1931\/revisions\/113698"}],"up":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1906"}],"wp:attachment":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/media?parent=1931"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/categories?post=1931"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/tags?post=1931"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}