{"id":1452,"date":"2017-08-26T11:54:02","date_gmt":"2017-08-26T16:54:02","guid":{"rendered":"http:\/\/coevolving.com\/blogs\/?p=1452"},"modified":"2017-08-26T11:54:02","modified_gmt":"2017-08-26T16:54:02","slug":"learning-data-science-hands-on","status":"publish","type":"post","link":"https:\/\/coevolving.com\/blogs\/learning-data-science-hands-on\/","title":{"rendered":"Learning data science, hands-on"},"content":{"rendered":"<p>For the <em>Quantitative Methodologies for Design Research<\/em> (\u5b9a\u91cf\u7814\u7a76\u65b9\u6cd5) course for <a href=\"http:\/\/tjdi.tongji.edu.cn\/about.do?ID=29&amp;lang=_en\">Ph.D. students at Tongji University<\/a> in spring 2017, <a href=\"https:\/\/nousala.com\/talo\/\">Susu Nousala<\/a> invited me to join the team of instructors in collaborative education in Shanghai. \u00a0Experts were brought in during the course to guide the graduate students.<\/p>\n<p>My participation in the course over two days had three parts: \u00a0(<a href=\"#preparing\">a<\/a>) preparing a lecture outline; (<a href=\"#orienting\">b<\/a>) orienting the students; and (<a href=\"#equipping\">c<\/a>) equipping the students with tools.<\/p>\n<h3 id=\"preparing\">(A) Preparing a lecture outline<\/h3>\n<p>While I&#8217;m comfortable with the mathematics underlying statistical analysis, I have a lot of practical experience of working with business executives who aren&#8217;t. \u00a0Thus, my approach to working with data relies a lot on presentation graphics to defog the phenomena. \u00a0While the label of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Data_science#History\"><em>data science<\/em> began to rise circa 2012<\/a>, I&#8217;ve had the benefit of practical experience that predates that.<\/p>\n<figure id=\"attachment_1455\" aria-describedby=\"caption-attachment-1455\" style=\"width: 396px\" class=\"wp-caption alignright\"><a href=\"http:\/\/www.softwarepreservation.org\/projects\/apl\/Brochures\/APL2SalesBrochure\/view\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1455 size-full\" src=\"http:\/\/coevolving.com\/blogs\/wp-content\/uploads\/2017\/08\/1994_IBM_GC26-8237-00.png\" alt=\"Today's APL\" width=\"396\" height=\"296\" \/><\/a><figcaption id=\"caption-attachment-1455\" class=\"wp-caption-text\">AGSS: A Graphical Statistical System (1994)<\/figcaption><\/figure>\n<p>In my first professional assignment in IBM Canada in 1985, <em>data science<\/em> would have been called <a href=\"https:\/\/en.wikipedia.org\/wiki\/Econometrics\"><em>econometrics<\/em><\/a>. \u00a0My work included forecasting country sales, based on price-performance indexes (from the mainframe, midrange and personal computer product divisions) and economic outlooks from Statistics Canada. \u00a0Two years before the <em>Macintosh II<\/em> would bring color to personal computing, I was an early adopter of <em>GRAFSTAT<\/em>: &#8220;<a href=\"http:\/\/dx.doi.org\/10.1145\/384283.801082\">An APL system for interactive scientific-engineering graphics and data analysis<\/a>&#8221; developed at IBM Research. \u00a0This would eventually become an IBM program product by called <em><a href=\"ftp:\/\/ftp.software.ibm.com\/ps\/products\/apl2\/info\/agss.txt\">AGSS<\/a><\/em> (A Graphical Statistical System) by 1994.<\/p>\n<figure style=\"width: 396px\" class=\"wp-caption alignright\"><a href=\"http:\/\/spot.pcc.edu\/~mtrigobo\/personal\/metaphor.html\"><img decoding=\"async\" class=\"size-full\" src=\"http:\/\/spot.pcc.edu\/~mtrigobo\/personal\/metaphor1.png\" alt=\"Metaphor Computer Systems workstation\" width=\"396\" \/><\/a><figcaption class=\"wp-caption-text\">Metaphor Computer Systems workstation<\/figcaption><\/figure>\n<p>In 1988, I had an assignment where <em>data science<\/em> would have been called <a href=\"https:\/\/en.wikipedia.org\/wiki\/Marketing_science\"><em>marketing science<\/em><\/a>. \u00a0I was sent to California to work in the IBM partnership with <a href=\"https:\/\/en.wikipedia.org\/wiki\/Metaphor_Computer_Systems\"><em>Metaphor Computer Systems<\/em><\/a>. This was a <a href=\"https:\/\/en.wikipedia.org\/wiki\/PARC_(company)\">Xerox PARC<\/a> spin-off with a vision that predated <a href=\"https:\/\/en.wikipedia.org\/wiki\/World_Wide_Web#History\">the first web page on the World Wide Web<\/a> by a few years. \u00a0These activities led me into the <a href=\"https:\/\/doi.org\/10.1287\/inte.20.1.151\">TIMS Marketing Science Conference in 1990<\/a>, cofounding the <a href=\"http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.198.4937\">Canadian Centre for Marketing Information Technologies (C<sup>2<\/sup>MIT)<\/a> and contributing <a href=\"http:\/\/coevolving.com\/commons\/1994_MktgInfoRev_Chap4_Ing\">chapters to <em>The Marketing Information Revolution<\/em> published in 1994<\/a>.<\/p>\n<p>This journey led me to appreciate the selection and use of computer-based tools for quantitative analysis. \u00a0Today, the two leading platforms in &#8220;<a href=\"https:\/\/cognitiveclass.ai\/courses\/data-science-101\/\">Data Science 101<\/a>&#8221; are\u00a0<em><a href=\"https:\/\/wiki.python.org\/moin\/BeginnersGuide\/Programmers\">Python<\/a><\/em>\u00a0(a general purpose language with statistical libraries), and the\u00a0<em><a href=\"https:\/\/www.r-project.org\/\">R Project for Statistical Computing<\/a>\u00a0<\/em>(a specialized package for data analysis and visualization). \u00a0Both are open source projects, and free to download and use on personal computers. \u00a0I tried both. \u00a0<em>R<\/em>\u00a0is a <a href=\"https:\/\/en.wikipedia.org\/wiki\/High-level_programming_language\">higher level programming language<\/a>\u00a0more similar to the <a href=\"https:\/\/en.wikipedia.org\/wiki\/APL_(programming_language)\"><em>APL<\/em> programming language<\/a>\u00a0that gets work done more quickly. \u00a0For statistical work, I recommend <em>R<\/em> over <em>Python<\/em> (although\u00a0<a href=\"https:\/\/scottlocklin.wordpress.com\/2013\/07\/28\/ruins-of-forgotten-empires-apl-languages\/\"><em>APL<\/em> is a theoretically better implementation<\/a>).<\/p>\n<figure style=\"width: 396px\" class=\"wp-caption alignright\"><a href=\"https:\/\/www.flickr.com\/photos\/daviding\/32678425810\"><img decoding=\"async\" class=\"size-full\" src=\"https:\/\/farm1.staticflickr.com\/267\/32678425810_56ef2b2098_z_d.jpg\" alt=\"Intro to R Programming, Big Data University\" width=\"396\" \/><\/a><figcaption class=\"wp-caption-text\">Intro to R Programming, Big Data University, Feb. 22, 2017<\/figcaption><\/figure>\n<p>Since I live in Toronto, I attended the February session of\u00a0<a href=\"https:\/\/www.meetup.com\/Cognitive-Toronto\/events\/237248880\/\"><em>Data Science with R \u2013 Bootcamp<\/em><\/a>\u00a0in person, at Ryerson University. \u00a0There, I was watched\u00a0<a href=\"http:\/\/www.polonglin.com\/\">Polong Lin<\/a> leading a class through <em>R<\/em> using the <a href=\"http:\/\/jupyter.org\/about.html\"><em>Jupyter<\/em><\/a> notebook, both in (i) an interactive version, and (ii) a printable version. \u00a0Students had the choice to either follow Polong (i) actively, in a step-by-step execution in the <em>Cognitive Class Virtual Lab<\/em> (formerly called the <a href=\"https:\/\/datascientistworkbench.com\/\"><em>Data Scientist Workbench<\/em><\/a>) with a cloud-based <em>R<\/em> session through their web browsers, or (ii) passively, reading the static printable content.<\/p>\n<p><!--more--><\/p>\n<p>Polong was helpful in guiding us with course resources that would be available in Shanghai. \u00a0In North America, we have <a href=\"https:\/\/cognitiveclass.ai\/\">cognitiveclass.ai<\/a>\u00a0with course materials, and\u00a0<a href=\"https:\/\/datascientistworkbench.com\/\">datascientistworkbench.com<\/a>\u00a0as a cloud computing platform (that includes <em>R<\/em> and <em>Python<\/em>). \u00a0For China, there are parallel sites at <a href=\"https:\/\/bigdatauniversity.com.cn\/\">bigdatauniversity.com.cn<\/a> and <a href=\"https:\/\/datascientistworkbench.cn\/\">datascientistworkbench.cn<\/a>\u00a0in Chinese, that native speakers could be more comfortable using.<\/p>\n<p>Based on Polong&#8217;s materials, I developed a Jupyter notebook additionally emphasizing <a href=\"http:\/\/ggplot2.org\/\">graphical presentation<\/a> for an in-person lecture in Shanghai. \u00a0These materials are cached at\u00a0<a href=\"http:\/\/coevolving.com\/tongji\/201704_DataScienceR\/\">coevolving.com\/tongji\/201704_DataScienceR\/<\/a>, where they are accessible globally.<\/p>\n<h3 id=\"orienting\">(B) Orienting the students<\/h3>\n<p>Ph.D. students in design will not have been required to have studied mathematics at the university level. \u00a0However, I then recalled that Shanghai high school students had a history with PISA achievement of &#8220;<a href=\"http:\/\/www.telegraph.co.uk\/news\/worldnews\/asia\/china\/10494678\/PISA-education-tests-Why-Shanghai-pupils-are-so-special.html\">top of the global class in maths with an average score &#8230; or the equivalent of nearly three years of schooling, above the average<\/a>&#8221; in 2013.<\/p>\n<p><a href=\"https:\/\/www.flickr.com\/photos\/daviding\/33849750915\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full\" src=\"https:\/\/farm3.staticflickr.com\/2910\/33849750915_b7861aa711_z_d.jpg\" alt=\"Ph.D. course in Quantitative Methods, Tongji University College of Design and Innovation\" width=\"640\" height=\"360\" \/><\/a><br \/>\nThis strength flavoured the description of my approach for the lecture to students:<\/p>\n<ul>\n<li>Did you study <a href=\"https:\/\/en.wikipedia.org\/wiki\/Linear_algebra\">linear algebra<\/a>? \u00a0(Pause to have a Chinese professor translate that.)<\/li>\n<li>Of course, you studied <a href=\"http:\/\/wenr.wes.org\/2012\/07\/wenr-junejuly-2012-senior-secondary-mathematics-education-in-china\">linear algebra in high school<\/a>.<\/li>\n<li>So, how do I approach exploratory data analysis?<\/li>\n<li>I plot the data, and draw a straight line through it.<\/li>\n<li>If the plot doesn&#8217;t look right, I move the data so that straight line looks right.<\/li>\n<\/ul>\n<p><a href=\"http:\/\/coevolving.com\/tongji\/201704_DataScienceR\/\"><img decoding=\"async\" class=\"alignright size-medium wp-image-1464\" src=\"http:\/\/coevolving.com\/blogs\/wp-content\/uploads\/2017\/08\/201704_Tongji_OrientationDemonstration-416x640.png\" alt=\"An Orientation Demonstration (Jupyter notebook)\" width=\"300\" srcset=\"https:\/\/coevolving.com\/blogs\/wp-content\/uploads\/2017\/08\/201704_Tongji_OrientationDemonstration-416x640.png 416w, https:\/\/coevolving.com\/blogs\/wp-content\/uploads\/2017\/08\/201704_Tongji_OrientationDemonstration.png 542w\" sizes=\"(max-width: 416px) 100vw, 416px\" \/><\/a><\/p>\n<p>The workshop scope was then explained as:<\/p>\n<ul>\n<li><em>NOT<\/em> to teach you everything about data science!<\/li>\n<li>You should know about ways to represent data to support your research findings.<\/li>\n<li>You will see some good tools for qualitative methods (and maybe even quantitative methods).<\/li>\n<li>We can work together to get you started on tools that will suit your needs.<\/li>\n<\/ul>\n<p>Working with the end in mind, the orientation stepped through a recommended package with three tools: \u00a0(i) <a href=\"http:\/\/ggplot2.org\/\"><em>ggplot2<\/em><\/a>, through (ii) <a href=\"http:\/\/jupyter.org\/\"><em>Jupyter<\/em><\/a>, on (iii) <a href=\"https:\/\/www.r-project.org\/\"><em>R<\/em><\/a>.<\/p>\n<p>The demonstration loaded an example of\u00a0Housing, with <a href=\"https:\/\/vincentarelbundock.github.io\/Rdatasets\/doc\/Ecdat\/Housing.html\">Sales Prices of Houses in the City of Windsor<\/a> as 546 rows and 12 variables. \u00a0This showed that a higher housing price correlates with a larger lot size, something that makes sense intuitively. \u00a0The ggplot2 library has a nice feature of <a href=\"http:\/\/ggplot2.tidyverse.org\/reference\/facet_grid.html#examples\">facet grids<\/a>, so that incorporating the number of bedrooms into an analysis can be visualized as a collection of plots (rather than a single plot).<\/p>\n<h3 id=\"equipping\">(C) Equipping the students with tools<\/h3>\n<p>Having then shown the students what outputs could look like, we released them to experiment as a self-study group, with the professors available as standby resources. \u00a0As Ph.D. students at one of the top universities in China, they were expected to step up to the challenge. \u00a0Since neither Susu nor I are proficient in Mandarin, the students would likely learn faster as a group if not slowed down by second-language conversion.<\/p>\n<p>The students then enthusiastically set upon getting their tools to work. \u00a0They collectively surfaced two challenges: \u00a0(i) technical; and (ii) research use.<\/p>\n<p>(i) The <em>technical issue<\/em> was that, although the\u00a0<a href=\"https:\/\/bigdatauniversity.com.cn\/\">bigdatauniversity.com.cn<\/a>\u00a0web site was readily accessible, the students found the cloud-based <a href=\"https:\/\/datascientistworkbench.cn\/\">datascientistworkbench.cn<\/a>\u00a0slow over the Internet. \u00a0While we in North America expect high bandwidth speeds in our workplaces and homes, the responsiveness through the browser interface at Tongji University to the cloud platform was too slow. \u00a0This led the students to prefer installing open source software tools onto their personal computers, dissolving Internet connectivity problems.<\/p>\n<p>Most of the students were running MacBooks, and a few were on Windows laptops. \u00a0Downloading <a href=\"https:\/\/cran.r-project.org\/mirrors.html\">R from the Comprehensive R Archive Network<\/a>\u00a0and installing that program was easy. \u00a0Downloading <a href=\"https:\/\/docs.continuum.io\/anaconda\/install\/\">Anaconda from Continuum Analytics<\/a>\u00a0as a step towards a <em>Jupyter<\/em> notebook was relatively straightforward. \u00a0But then, with both\u00a0<em>Anaconda<\/em>\u00a0and <em>R<\/em>\u00a0up and running, getting <em>Jupyter<\/em> to connect to <em>R<\/em> was a challenge. \u00a0This required opening up a <em>Terminal<\/em> &#8212; a new experience for most <a href=\"https:\/\/en.wikipedia.org\/wiki\/Terminal_(macOS)\">on MacOS<\/a>, but somewhat more familiar as <a href=\"https:\/\/en.wikipedia.org\/wiki\/Cmd.exe\">command line on Windows<\/a>\u00a0&#8212; and typing in magic incantations.<\/p>\n<ul>\n<li>The installation of <em><a href=\"https:\/\/irkernel.github.io\/installation\/\">IRKernel<\/a><\/em> is not a drag-and-drop activity, and sometimes requires more technical knowledge to interpret the messages signalling other than success.<\/li>\n<li>MacOS X with <em>Jupyter<\/em> needs some <em>Python<\/em>\u00a0libraries that requires the installation of <em>XCode<\/em>.<\/li>\n<li>MacOS X <a href=\"https:\/\/support.apple.com\/en-ca\/HT201341\">after 10.5 (Leopard) no longer includes X11<\/a>, requiring a separate installation of <em>XQuartz<\/em>.<\/li>\n<li>On MacOS X, installing the <em>IRKernel<\/em> would not work if <em>R<\/em> was launched by clicking on the <em>R<\/em> icon, but would if <em>R<\/em> was started from a <em>Terminal<\/em>.<\/li>\n<\/ul>\n<p>Resolving these issues had the Ph.D. students collectively helping each other for 6 hours in the afternoon and evening on the first course day, and then 2 hours on the second morning. \u00a0Updating MacOS X with XCode over the Internet was slow on the university network. \u00a0On Windows, one student easily got all of the packages working on his computer, and then struggled to help a friend struggling with similar hardware and software.<\/p>\n<p>The ordeal of installing software had an unexpected benefit of becoming a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Team_building\">team-building<\/a> activity. \u00a0The students banded together in mutual support!<\/p>\n<p>(ii) The <em>research use<\/em> issue arose as the students worked their way through the Big Data University exercises. \u00a0Exercises typically use datasets already prepackaged for use, \u00a0so students can focus on the programming language. \u00a0Real research projects requiring bringing in real datasets. \u00a0Pointing out the <a href=\"https:\/\/en.wikibooks.org\/wiki\/R_Programming\/Importing_and_exporting_data\">import features for R programming<\/a> moved the students one step ahead.<\/p>\n<p>In\u00a0a course oriented towards a future of big data, this raised the question: \u00a0where is the data? \u00a0The phenomenon of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Open_data\">open data<\/a> &#8212; as electronic data sources readily accessible over the Internet &#8212; is nascent within the People&#8217;s Republic of China. \u00a0This led to questions about <a href=\"https:\/\/en.wikipedia.org\/wiki\/Research_design\">research design<\/a>, where students would have to determine questions of interest, and plan ways to collect data.<\/p>\n<p>After those two days, my participation in the course diminished with a few exchanges over WeChat. \u00a0Susu brought some of the students onto a research project in Southern China, putting their research design learning into practice over the summer. \u00a0By the fall, the students should have performed preliminary data analysis, so that coaching may be welcomed on finer points about statistics.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Collaborating for two days in the Quantitative Methodologies for Design Research (\u5b9a\u91cf\u7814\u7a76\u65b9\u6cd5) course at Tongji University led to dealing with practical issues of technology and research design.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[8],"tags":[466,467,468],"class_list":["post-1452","post","type-post","status-publish","format-standard","hentry","category-universities","tag-big-data","tag-quantitative-methods","tag-tongji-university"],"_links":{"self":[{"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/posts\/1452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/comments?post=1452"}],"version-history":[{"count":26,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/posts\/1452\/revisions"}],"predecessor-version":[{"id":1480,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/posts\/1452\/revisions\/1480"}],"wp:attachment":[{"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/media?parent=1452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/categories?post=1452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coevolving.com\/blogs\/wp-json\/wp\/v2\/tags?post=1452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}