{"id":13336,"date":"2018-04-10T13:46:54","date_gmt":"2018-04-10T04:46:54","guid":{"rendered":"https:\/\/avinton.com\/en\/?page_id=13336"},"modified":"2024-10-03T10:52:20","modified_gmt":"2024-10-03T01:52:20","slug":"machine-learning-engineer-skills-required","status":"publish","type":"page","link":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/","title":{"rendered":"Machine Learning Engineer Skills Required"},"content":{"rendered":"

[vc_row][vc_column][vc_column_text]<\/p>\n

Core Skills of a Machine Learning Engineer<\/span><\/span><\/h1>\n

[\/vc_column_text][\/vc_column][\/vc_row][vc_row][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Linux<\/span><\/span><\/h2>\n

As a starting point for a Machine Learning Engineer, we recommend at least <\/span><\/span>LPIC level 1<\/span><\/span><\/a>.<\/span><\/span><\/p>\n

This will ensure you don’t waste time trying to figure out what you should be typing on the command line to manipulate your environment.<\/span><\/span><\/p>\n

For engineers experienced in linux or MacOS command line this may not be required<\/span><\/span>[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13351″ img_size=”medium” alignment=”center” css=”.vc_custom_1523333989220{margin-top: 1px !important;background-position: 0 0 !important;background-repeat: no-repeat !important;}”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Python Programming<\/h2>\n

You should have a solid grounding in Python Programming.<\/p>\n

This will allow you to be more productive and concentrate on the machine learning task rather than troubleshooting syntax errors<\/p>\n

For a good comprehensive overview of the Python Language we recommend this Python 3 book<\/a>.<\/p>\n

You will likely be looking at Python Code every day so having a good understanding of it from the beginning is going to make you more effective and efficient.<\/p>\n

Think of Python as your best friend – don’t you want to know them better?<\/p>\n

[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13358″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Machine Learning Development Environment<\/h2>\n

The most commonly used environment for machine learning in Python is Jupyter Notebook & Anaconda<\/p>\n

Please find some links below how to set these up<\/p>\n

Jupyter Notebook<\/a>
\nAnaconda<\/a>
\nJupyter & Anaconda setup for Machine Learning in Python<\/a> (Avinton Academy)<\/p>\n

[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13362″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Machine Learning Basics<\/h2>\n

Python has some native machine learning capability and building up on your Python Programming Knowledge we recommend this Python Deep Learning book<\/a> for a gentle transition from programming to machine learning
\nHere are some introductory tutorials about Machine Learning Basics:<\/p>\n

ChainerRL<\/a> (Avinton Academy)
\nRandom Forest<\/a> (Avinton Academy)
\nNaive Bayes<\/a> (Avinton Academy)
\nClassification vs Regression<\/a> (Avinton Academy)
\nSVM<\/a> (Avinton Academy)
\n[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13366″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Machine Learning Libraries and Frameworks<\/h2>\n

Depending on the application and the environment a machine learning project may use different environment, libraries, frameworks that are best suited for the problem being solved.<\/p>\n

The most common ones today are:<\/p>\n

TensorFlow<\/a>
\nKeras<\/a>
\nScikit Learn<\/a><\/p>\n

You should try to familiarise yourself with these<\/p>\n

[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13381″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Data Analyses<\/h2>\n

Depending on the type of problem being solved you may be required to do some data processing and data analyses.<\/p>\n

SQL<\/h3>\n

In many large organisations data is stored in large Relational Databases. We use SQL for querying and manipulating the data in the database. SQL is a very powerful language where we can perform some rather complex data analyses with only a few lines of simple code. You are encouraged to familiarise yourself with SQL using PostgreSQL database<\/a> as it is the most advanced Open Source Database.<\/p>\n

Python – Numpy<\/h3>\n

During the data pre-processing for Machine Learning we often need to do some data extraction and manipulation from the original dataset. Python Numpy<\/a> is a package that gives quite extensive and advanced mathatics functionality that can be used for all kinds of data modelling and analyses.<\/p>\n

You are encouraged to familiarise yourself with the numpy’s capability by browsing through the NUMPY API reference<\/a><\/p>\n

Pandas<\/h3>\n

Pandas<\/a> is a library for working with data structures in Python. It extends the capability of Numpy.
\nPandas API Reference<\/a><\/p>\n

Have a look at the documentation and see which functions may be useful for your data manipulation requirements.<\/p>\n

[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13379″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

OpenCV – Image Processing<\/span><\/span><\/h2>\n

Many Machine Learning Engineers and projects in the industry today are using AI for object detection and other related image analyses.<\/span><\/span><\/p>\n

For any of such projects we usually use OpenCV integrated within a machine learning library to label the images and do any pre-processing if required<\/span><\/span><\/p>\n

You can find some OpenCV tutorials below:<\/span><\/span><\/p>\n

Python OpenCV setup<\/a> (Avinton Academy)
\nOpenCV simple Exercise<\/a> (Avinton Academy)
\nOpenCV Advanced Exercise<\/a> (Avinton Academy)[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13372″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Online Courses<\/h2>\n

Stanford Machine Learning – Coursera – Andrew Ng<\/a>
\nLecture Series is available on YouTube<\/a><\/p>\n

Coursera Deep Learning<\/a><\/p>\n

FastAI<\/a> have a good set of courses which are free especially their Computational Linear Algebra course<\/a>.<\/p>\n

Lecture Series<\/h3>\n

Deep RL Bootcamp<\/a>
\nAndrej Karpathy\u2019s CNN course at Stanford<\/a> (CS231n: Convolutional Neural Networks for Visual Recognition)
\nSergey Levine\u2019s Deep Reinforcement Learning Course<\/a> – UC Berkeley (CS 294: Deep Reinforcement Learning)
\nLearning Machines 101<\/a> by Richard Golden<\/p>\n

[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13394″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row][vc_row content_placement=”middle”][vc_column width=”2\/3″][vc_column_text]<\/p>\n

Infrastructure Basics<\/span><\/span><\/h2>\n

Machine Learning tasks are typically rather computationally intensive so it is important that a Machine Learning Engineer understands some infrastructure basics in order to take full advantage of the underlying server hardware in our environment to make our model training process efficient.<\/span><\/span><\/p>\n

Server Resources: CPU \/ Disk \/ RAM<\/span><\/span><\/h3>\n

Avinton Academy instructor led training will give you the basics on this area. Please attend the next available session when you have time.<\/span><\/span><\/p>\n

CPU vs GPU<\/span><\/span><\/h3>\n

In many cases we use a GPU to train the machine learning model more efficiently. It will be good to familiarise yourself with how a GPU works and why it is better at certain tasks than a CPU<\/span><\/span><\/p>\n

This is covered in the Avinton Academy Infrastructure workshop day 2<\/span><\/span><\/p>\n

Virtualization Concepts<\/span><\/span><\/h3>\n

In most cases your development environment is likely going to be on a virtual machine. As such it is good for you to familiarise yourself with Virtualisation concepts. The best approach for this would be to set up VMWare ESXI on a server from scratch and create the host Virtual Machines using the Hypervisor’s web gui.<\/span><\/span><\/p>\n

This is covered in Avinton Academy Infrastructure Workshop.<\/span><\/span><\/p>\n

AWS EC2<\/span><\/span><\/h3>\n

Many AI projects are deployed in the cloud. AWS (Amazon Web Services) is the leader in cloud services and as such we recommend you familiarise yourself with working with their EC2 platform (Elastic Compute).<\/span><\/span><\/p>\n

You can refer to <\/span><\/span>AWS’s official documentation<\/span><\/span><\/a> and on Avinton Academy we have a dedicated section guiding you through <\/span><\/span>how to work with AWS<\/span><\/span><\/a><\/p>\n

Docker Containers<\/span><\/span><\/h3>\n

We often use Docker containers to easily replicate our environment from one system to another. As such I recommend you familiarise yourself with Docker Containers and try to Dockerise your environment for sharing with other team members.<\/span><\/span>[\/vc_column_text][\/vc_column][vc_column width=”1\/3″][vc_single_image image=”13373″ img_size=”medium” alignment=”center”][\/vc_column][\/vc_row]<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"

[vc_row][vc_column][vc_column_text] Core Skills of a Machine Learning Engineer [\/vc_column_text][\/vc_column][\/vc_row][vc_row][vc_column width=”2\/3″][vc_column_text] Linux As a starting point for a Machine Learning Engineer, we recommend at least LPIC level 1. This will ensure you don’t waste time trying to figure out what you should be typing on the command line to manipulate your environment. For engineers experienced in
Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":13374,"parent":1906,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-13336","page","type-page","status-publish","has-post-thumbnail","hentry"],"yoast_head":"\nMachine Learning Engineer Skills Required - Avinton Japan<\/title>\n<meta name=\"description\" content=\"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning Engineer Skills Required - Avinton Japan\" \/>\n<meta property=\"og:description\" content=\"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/\" \/>\n<meta property=\"og:site_name\" content=\"Avinton Japan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Avintons\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-03T01:52:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"824\" \/>\n\t<meta property=\"og:image:height\" content=\"501\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@AvintonJapan\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/\",\"url\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/\",\"name\":\"Machine Learning Engineer Skills Required - Avinton Japan\",\"isPartOf\":{\"@id\":\"https:\/\/avinton.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg\",\"datePublished\":\"2018-04-10T04:46:54+00:00\",\"dateModified\":\"2024-10-03T01:52:20+00:00\",\"description\":\"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.\",\"breadcrumb\":{\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage\",\"url\":\"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg\",\"contentUrl\":\"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg\",\"width\":824,\"height\":501,\"caption\":\"Avinton machine learning need to know\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/avinton.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Avinton Academy\",\"item\":\"https:\/\/avinton.com\/en\/academy\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Machine Learning Engineer Skills Required\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/avinton.com\/en\/#website\",\"url\":\"https:\/\/avinton.com\/en\/\",\"name\":\"Avinton Japan\",\"description\":\"Tailored Solutions and Consulting in AI and Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/avinton.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning Engineer Skills Required - Avinton Japan","description":"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning Engineer Skills Required - Avinton Japan","og_description":"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.","og_url":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/","og_site_name":"Avinton Japan","article_publisher":"https:\/\/www.facebook.com\/Avintons\/","article_modified_time":"2024-10-03T01:52:20+00:00","og_image":[{"width":824,"height":501,"url":"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@AvintonJapan","twitter_misc":{"Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/","url":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/","name":"Machine Learning Engineer Skills Required - Avinton Japan","isPartOf":{"@id":"https:\/\/avinton.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage"},"image":{"@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage"},"thumbnailUrl":"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg","datePublished":"2018-04-10T04:46:54+00:00","dateModified":"2024-10-03T01:52:20+00:00","description":"What a machine learning engineer needs to know to be productive and effective in an AI project in today's active job market.","breadcrumb":{"@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#primaryimage","url":"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg","contentUrl":"https:\/\/avinton.com\/wp-content\/uploads\/2018\/04\/machine-learning-need-to-know.jpg","width":824,"height":501,"caption":"Avinton machine learning need to know"},{"@type":"BreadcrumbList","@id":"https:\/\/avinton.com\/en\/academy\/machine-learning-engineer-skills-required\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/avinton.com\/en\/"},{"@type":"ListItem","position":2,"name":"Avinton Academy","item":"https:\/\/avinton.com\/en\/academy\/"},{"@type":"ListItem","position":3,"name":"Machine Learning Engineer Skills Required"}]},{"@type":"WebSite","@id":"https:\/\/avinton.com\/en\/#website","url":"https:\/\/avinton.com\/en\/","name":"Avinton Japan","description":"Tailored Solutions and Consulting in AI and Big Data","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/avinton.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/13336","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/comments?post=13336"}],"version-history":[{"count":37,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/13336\/revisions"}],"predecessor-version":[{"id":113706,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/13336\/revisions\/113706"}],"up":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/pages\/1906"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/media\/13374"}],"wp:attachment":[{"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/media?parent=13336"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/categories?post=13336"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/avinton.com\/en\/wp-json\/wp\/v2\/tags?post=13336"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}