{"id":1347,"date":"2021-09-16T19:59:05","date_gmt":"2021-09-16T17:59:05","guid":{"rendered":"https:\/\/dbdmg.polito.it\/dbdmg_web\/?p=1347"},"modified":"2022-09-13T10:17:21","modified_gmt":"2022-09-13T08:17:21","slug":"big-data-architectures-and-data-analytics-2021-2022","status":"publish","type":"post","link":"https:\/\/dbdmg.polito.it\/dbdmg_web\/2021\/big-data-architectures-and-data-analytics-2021-2022\/","title":{"rendered":"Big Data: Architectures and Data Analytics (2021\/2022)"},"content":{"rendered":"\n<h2 class=\"eplus-wrapper wp-block-heading\" id=\"general-information\">General Information<\/h2>\n\n\n\n<p class=\" eplus-wrapper\"><strong>SSD<\/strong>: ING-INF\/05<\/p>\n\n\n\n<p class=\" eplus-wrapper\"><strong>CFU<\/strong>: 6<\/p>\n\n\n\n<p class=\" eplus-wrapper\"><strong>Professor<\/strong>: Paolo Garza<\/p>\n\n\n\n<p class=\" eplus-wrapper\"><strong>Teaching Assistant<\/strong>: Luca Colomba<\/p>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<h3 class=\"eplus-wrapper wp-block-heading\" id=\"announcements\">Announcements<\/h3>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-e83e1b\"><li>16-09-21: The first lecture is scheduled for September  27, 2021 at 16:00 in Classroom R1 <\/li><li>23-09-21: No lab activities during the first two weeks of the course<\/li><li>27-09-21: I created a page for &#8220;01QYDOV &#8211; Big data: architectures and data analytics&#8221; on Piazza: <a rel=\"noreferrer noopener\" href=\"https:\/\/piazza.com\/polito.it\/fall2021\/01qydov\" target=\"_blank\">piazza.com\/polito.it\/fall2021\/01qydov<\/a><br>Piazza is a Q&amp;A system that can be used to manage questions and answers offline. You can use it, instead of the email, when you have questions that are of interest also for other students (e.g., questions on the proposed solutions, problems with the configuration of the used software, etc.). There are different &#8220;channels&#8221; (lectures, lab, others) related to different topics. You can publish both public or private questions.<br>I will answer periodically to your questions (I will try to answer them daily).<\/li><li>10-10-21: First lab activity this week<ul><li>Monday, October 11, 17:30 &#8211; 19:00 &#8211; LAIB1 &#8211; TEAM 1<\/li><li>Wednesday, October 13, 14:30 &#8211; 16:00 &#8211; LAIB1 &#8211; TEAM 2<\/li><\/ul><\/li><li>10-10-21: <strong>The lecture scheduled for Tuesday, October 12<\/strong> at 10:00 <strong>will start at  11:30 and will last only 1.5 hours<\/strong> (from 11:30 to 13:00)<\/li><\/ul>\n\n\n<hr class=\"wp-block-separator has-css-opacity eplus-wrapper\"\/>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<h3 class=\"eplus-wrapper wp-block-heading\" id=\"teaching-material\">Teaching Material<\/h3>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-e5272f\"><li>Introduction to the course content and exam rules (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/00_Intro_BigData_2122.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>Introduction to Big Data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/01_Intro_BigData_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>Big Data Architectures (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/02_Architectures_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>Hadoop and MapReduce <ul><li>Introduction to Apache Hadoop and the MapReduce programming paradigm (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/03_Intro_HadoopAndMapReduce_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>) <ul><li>Interaction with HDFS and Hadoop by means of the command line (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/03b_HDFS_Hadoop_CommandLine_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><\/li><li>Hadoop implementation of MapReduce (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/04_HadoopImplementationOfMapReduce_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>BigData@Polito environment + Jupyter \u2013 How to submit MapReduce jobs on BigData@Polito (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/04b_ClusterJupyter_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><\/li><\/ul><ul><li>MapReduce and Hadoop \u2013 Advanced Topics: Multiple inputs, Multiple outputs, Distributed cache (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/06_AdvancedTopicsMapReduce_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>MapReduce \u2013 Design patterns \u2013 Part 1 (slides) (slides without black background) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/05_MapReduce_Patterns_Part1_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>MapReduce \u2013 Design patterns \u2013 Part 2 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/07_MapReduce_Patterns_Part2_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><ul><li>MapReduce \u2013 Relational Algebra\/SQL operators (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/08_SQLOperators_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><\/li><li>Spark<ul><li>Introduction to Apache Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/10_SparkIntroduction_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>How to submit Spark applications (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/10b_SparkSubmit_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><\/li><li>RDD-based programs RDDs<ul><li>Creation, basic transformations and actions (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/11_SparkRDD_Basic_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><\/ul><ul><li>Key-value pair RDDs: transformations and actions on PairRDDs (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/12_SparkRDD_PairRDD_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>DoubleRDDs (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/13_SparkRDD_DoubleRDD_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<\/li><li>Advanced Topics: Cache, accumulators, broadcast variables (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/14_SparkRDD_AdvancedTopics_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>) &#8211; <strong><span style=\"color:#f53004\" class=\"has-inline-color\">Updated on November 6. 2021<\/span><\/strong><\/li><\/ul><\/li><li>Spark SQL, Datasets and DataFrames (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/15_SparkSQL_Datasets_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Spark SQL &#8211; Join examples (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/ExamplesSparkSQLJoins.zip\" target=\"_blank\">ExamplesSparkSQLJoins.zip<\/a>)<\/li><\/ul><\/li><li>Data Mining<ul><li>Recap data mining tasks (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/DMintro.pdf\" target=\"_blank\">slides<\/a>) &#8211; From the &#8220;Data Science And Database Technology&#8221; course<\/li><\/ul><\/li><\/ul><ul><li>Spark MLlib<ul><li>Introduction and Classification of structured data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/16_SparkMLlib_Part1_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Logistic Regression example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineLogisticRegression.zip\" target=\"_blank\">zip<\/a>)<\/li><li>Decision Trees example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineDecisionTree.zip\" target=\"_blank\">zip<\/a>)<\/li><li>Decision Trees and Categorical class label example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineDecisionTreeCategoricalLabel.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><li>Classification of textual data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/17_SparkMLlib_Part2_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Textual data classification example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineText.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><li>Classification and Parameter tuning (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/18_SparkMLlib_Part3_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Parameter tuning example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineLogisticRegressionCrossValidation.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><li>Clustering of structured data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/19_SparkMLlib_Part4_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Clustering example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineClustering.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><li>Itemset and Association rule mining (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/20_SparkMLlib_Part5_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Itemset and Association rule mining example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibFPGrowth.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><li>Linear regression (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/21_SparkMLlib_Part6_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Linear regression example code (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/MLlibPipelineLinearRegression.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Spark Streaming (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/22_SparkStreaming_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Examples: Word Count \u2013 Streaming versions (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/ExamplesSparkStreaming.zip\" target=\"_blank\" rel=\"noreferrer noopener\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<h4 class=\"eplus-wrapper wp-block-heading\" id=\"exercise\">Exercise<\/h4>\n\n\n\n<p class=\" eplus-wrapper\"><mark style=\"background-color:rgba(0, 0, 0, 0);color:#fc0303\" class=\"has-inline-color\">If you use your PC to write and run your code, import the projects based on Maven (those projects can be run locally).<br>If you use the PC available in the LAB, import the Eclipse projects with libraries (those projects cannot be run locally but only on the cluster exporting the jar file of the project).<\/mark><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-accent-2-color\"> <\/mark><\/p>\n\n\n\n<p class=\" eplus-wrapper\"> <\/p>\n\n\n<ul class=\"eplus-ce5B3z eplus-wrapper wp-block-list eplus-styles-uid-d8477d\"><li>MapReduce <ul><li>MapReduce exercises (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/01_MapReduce_Exercises_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Solutions of Exercises 1-29 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/SolutionsExMapReduce.zip\" target=\"_blank\">SolutionsExMapReduce.zip<\/a>)<\/li><\/ul><\/li><li>Basic project <ul><li>Linux and MacOs<ul><li>Basic Eclipse project for MapReduce applications (with libraries) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/WordCountLibraries.zip\" target=\"_blank\">MapReduceBasicProjectWithLibraries.zip<\/a>) &#8211; Import using Import\/General\/Existing Projects into Workspace<\/li><li>Basic Eclipse project for MapReduce applications (based on maven) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/MapReduceBasicProject.zip\" target=\"_blank\">MapReduceBasicProject.zip<\/a>) &#8211; Import it using Import\/Maven\/Existing Maven Projects<\/li><\/ul><\/li><li>Windows<ul><li>Basic Eclipse project for MapReduce applications (with libraries) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/WordCountLibraries.zip\" target=\"_blank\"><\/a><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/WordCountLibraries.zip\" target=\"_blank\">MapReduceBasicProjectWithLibraries<\/a>.<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/WordCountLibraries.zip\" target=\"_blank\">zip<\/a>) &#8211; Import using Import\/General\/Existing Projects into Workspace<\/li><li>Setup instructions for running MapReduce applications locally inside Eclipse (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/ConfigureWindowsEnviroment.pdf\" target=\"_blank\">ConfigureWindowsEnviroment.pdf<\/a>)<ul><li>You must install also <strong>JDK 1.8<\/strong> and select it for the imported project inside Eclipse. If you already installed the JDK environment\u00a0 but the version is greater than JDK 1.8 you must install also JDK 1.8.<\/li><li>Winutils executable (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/winutils.zip\" target=\"_blank\">winutils.zip<\/a>)<\/li><li>Basic Eclipse project for MapReduce applications (based on maven) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/09\/MapReduceBasicProjectWindows.zip\" target=\"_blank\">MapReduceBasicProjectWindows.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Spark<ul><li>Spark RDD-, Dataset-, DataFrame-based exercises (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/02_Spark_Exercises_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Example data \u2013 One folder with (few) data for each exercise (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/ExampleDataSpark.zip\" target=\"_blank\">ExampleDataSpark.zip<\/a>)<\/li><li>Solutions of Exercises 30-50 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/SolutionsSpark30-50.zip\" target=\"_blank\">SolutionsExSpark30-50.zip<\/a>) &#8211; Updated on November 19, 2021 &#8211; Added a second possibile solution for Exercise #44 (folder Exercise44 _v2)<\/li><li>Solutions of Exercises from 32 to 38 and 44 based on Spark sQL (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/SolSparkSQL32-38_44.zip\" target=\"_blank\">SolSparkSQL32-38_44.zip<\/a>)<\/li><\/ul><\/li><\/ul><ul><li>Spark streaming exercises (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/03_SparkStreaming_Exercises_BigData_NewStyle.pdf\" target=\"_blank\">slides<\/a>)<ul><li>Solutions of Exercises 51-53 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/SolutionsSparkStreaming51-53.zip\" target=\"_blank\">SolutionsSparkStreaming51_5<\/a><a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/SolutionsSparkStreaming51-53.zip\" target=\"_blank\" rel=\"noreferrer noopener\">3<\/a><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/SolutionsSparkStreaming51-53.zip\" target=\"_blank\">.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<hr class=\"wp-block-separator has-css-opacity eplus-wrapper\"\/>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<h3 class=\"eplus-wrapper wp-block-heading\" id=\"laboratory-material\">Laboratory Material<\/h3>\n\n\n\n<p class=\" eplus-wrapper\"><strong><span style=\"color:#fd0202\" class=\"has-inline-color\">No lab activities during the first two weeks of the course<\/span><\/strong><\/p>\n\n\n\n<p class=\" eplus-wrapper\">Team 1: Students from A to L &#8211; Monday from 17:30 to 19:00 &#8211; LAIB1<\/p>\n\n\n\n<p class=\" eplus-wrapper\">Team 2: Students from M to Z &#8211; Wednesday from 14:30 to 16:00 &#8211; LAIB1<\/p>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-4ca82b\"><li>Lab1: Hadoop and MapReduce<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab1_BigData.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Basic project and small example data set (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab1_BigData_with_libraries.zip\" target=\"_blank\">Lab1_BigData_with_libraries.zip<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux and macOS (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab1.zip\" target=\"_blank\">Lab1.zip<\/a>)<\/li><li>Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab1Windows.zip\" target=\"_blank\">Lab1Windows.zip<\/a>) <\/li><\/ul><\/li><li>Bigger data set: finefoods_text.txt (<a rel=\"noreferrer noopener\" href=\"https:\/\/www.dropbox.com\/s\/fswdiblx15mhmyo\/finefoods_text.zip?dl=0\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><\/ul><ul><li>Solution Bonus track<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab1_SolBonusMvn.zip\" target=\"_blank\">Lab1_SolBonusMvn.zip<\/a> &#8211; The project is based on mvn <\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-d84d54\"><li>Lab2: Filter with Hadoop MapReduce<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2_2021.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Hadoop \u2013 MapReduce (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2_Skeleton_with_libraries.zip\" target=\"_blank\">Lab2_Skeleton_with_libraries.zip<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux and macOS (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2_Skeleton.zip\" target=\"_blank\">Lab2_Skeleton.zip<\/a>)<\/li><li>Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2Windows_Skeleton.zip\" target=\"_blank\">Lab2Windows_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Outputs of the first lab (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/OutputFolderLab1.zip\" target=\"_blank\">OutputFolderLab1.zip<\/a>) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/OutputFolderLab1BonusTrack.zip\" target=\"_blank\">OutputFolderLab1BonusTrack.zip<\/a>). You can use them to test your application locally on your own PC if you are using Maven<\/li><li>Solution<ul><li> <a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2_Sol.zip\" target=\"_blank\" rel=\"noreferrer noopener\">Lab2_Sol.zip<\/a> &#8211; The project is based on mvn <\/li><\/ul><\/li><li>Solution Bonus track<ul><li><a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab2_SolBonus.zip\" target=\"_blank\" rel=\"noreferrer noopener\">Lab2_SolBonus.zip<\/a> &#8211; The project is based on mvn <\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-abd01b\"><li>Lab3: Frequently bought\/reviewed together application with Hadoop MapReduce <ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3_2021.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Hadoop \u2013 MapReduce (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3_Skeleton_with_libraries.zip\" target=\"_blank\">Lab3_Skeleton_with_libraries.zip<\/a>)<\/li><li>Sample file (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/AmazonTransposedDataset_Sample.txt\" target=\"_blank\">AmazonTransposedDataset_Sample.txt<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux and macOS (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3_Skeleton.zip\" target=\"_blank\">Lab3_Skeleton.zip<\/a>)<\/li><li>Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3Windows_Skeleton.zip\" target=\"_blank\">Lab3Windows_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3_Sol.zip\" target=\"_blank\">Lab3_Sol.zip<\/a> &#8211; The project is based on mvn <\/li><li>Comments on the three uploaded solutions (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab3_DraftSolution_BigData_NewStyle.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">slides<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-b5d91b\"><li>Lab4: Normalized ratings for product recommendations with Hadoop MapReduce<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab4_2021.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Hadoop \u2013 MapReduce (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab4_Skeleton_with_libraries.zip\" target=\"_blank\">Lab4_Skeleton_with_libraries.zip<\/a>)<\/li><li>Sample file (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/ReviewsSample.csv\" target=\"_blank\">ReviewsSample.csv<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux and macOS (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab4_Skeleton.zip\" target=\"_blank\">Lab4_Skeleton.zip<\/a>)<\/li><li>Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/10\/Lab4Windows_Skeleton.zip\" target=\"_blank\">Lab4Windows_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab4_Sol.zip\" target=\"_blank\">Lab4_Sol.zip<\/a> &#8211; The project is based on mvn <\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-7797cd\"><li>Lab5: Filter data and compute basic statistics with Apache Spark<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab5_2021.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab5_Skeleton_with_libraries.zip\" target=\"_blank\">Lab5_Skeleton_with_libraries.zip<\/a>)<\/li><li>Sample file (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/SampleLocalFile.csv\" target=\"_blank\">SampleLocalFile.csv<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab5BigData_Skeleton.zip\" target=\"_blank\">Lab5_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab5BigData_Sol.zip\" target=\"_blank\">Lab5_Sol.zip<\/a> <\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-e48884\"><li>Lab6: Frequently bought\/reviewed together application with Apache Spark<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab6_2023.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab6_Skeleton_with_libraries.zip\" target=\"_blank\">Lab6_Skeleton_with_libraries.zip<\/a>)<\/li><li>Sample file (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/ReviewsSample.csv\" target=\"_blank\">ReviewsSample.csv<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab6BigData_Skeleton.zip\" target=\"_blank\">Lab6_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Expected output \u2013 Task 1 (expected output if the input is the HDFS file Reviews.csv) (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/outputTask1Lab6.zip\" target=\"_blank\">outputTask1Lab6.zip<\/a>)<\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab6BigData_Sol.zip\" target=\"_blank\">Lab6_Sol.zip<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-92db83\"><li>Lab7: Bike sharing data analysis <ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab7_2023.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab7_Skeleton_with_libraries.zip\" target=\"_blank\">Lab7_Skeleton_with_libraries.zip<\/a>)<\/li><li>Sample data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/sampleData.zip\" target=\"_blank\">sampleData.zip<\/a>)<\/li><li>Example KML file (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/exampleKML.zip\" target=\"_blank\">exampleKML.zip<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab7_Skeleton.zip\" target=\"_blank\">Lab7_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><ul><li>Expected output <ul><li>Execution on sample data (sampleData\/registerSample.csv and sampleData\/stations.csv) and minimum criticality threshold = 0.4 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/resSampleData0.4-1.txt\" target=\"_blank\">part-00000<\/a>)<\/li><li>Execution on complete data (\/data\/students\/bigdata-01QYD\/Lab7\/register.csv and \/data\/students\/bigdata-01QYD\/Lab7\/stations.csv) and minimum criticality threshold = 0.6 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/resAllData0.6-1.txt\" target=\"_blank\">part-00000<\/a>)<\/li><\/ul><\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab7BigData_Sol.zip\" target=\"_blank\">Lab7_Sol.zip<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-a56ef4\"><li>Lab8: Bike sharing data analysis based on Spark SQL<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab8.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project \u2013 Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab8_Skeleton_with_libraries.zip\" target=\"_blank\">Lab8_Skeleton_with_libraries.zip<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/Lab8_Skeleton.zip\" target=\"_blank\">Lab8_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><ul><li>Sample data (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/11\/sampleData-1.zip\" target=\"_blank\">sampleData.zip<\/a>)<\/li><li>Solution<ul><li><a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab8_Sol.zip\" target=\"_blank\">Lab8_Sol.zip<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-424abe\"><li>Lab9: A classification pipeline with MLlib + SparkSQL<ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9.pdf\" target=\"_blank\">pdf<\/a>)<\/li><li>Skeleton Eclipse project \u2013 Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_Skeleton_with_libraries.zip\" target=\"_blank\">Lab9_Skeleton_with_libraries.zip<\/a>)<\/li><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_Skeleton.zip\" target=\"_blank\">Lab9_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Sample file with 100 reviews (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/ReviewsSample.csv\" target=\"_blank\">ReviewsSample.csv<\/a>)<\/li><li>Solution<ul><li>Logistic regression (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_SolLR.zip\" target=\"_blank\" rel=\"noreferrer noopener\">zip<\/a>)<\/li><li>DecisionTree (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_SolDT.zip\" target=\"_blank\" rel=\"noreferrer noopener\">zip<\/a>)<\/li><li>Logistic regression based on text analysis (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_SolLRText.zip\" target=\"_blank\" rel=\"noreferrer noopener\">zip<\/a>)<\/li><li>DecisionTree based on text analysis (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab9_SolDTText.zip\" target=\"_blank\" rel=\"noreferrer noopener\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<p class=\" eplus-wrapper\"><\/p>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-9c6fee\"><li>Lab10: Tweet analysis \u2013 Spark streaming <ul><li>Problem specification (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab10.pdf\" target=\"_blank\">pdf<\/a>)<\/li><\/ul><ul><li>Skeleton Eclipse project \u2013 Spark (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab10_Skeleton_with_libraries.zip\" target=\"_blank\">Lab10_Skeleton_with_libraries.zip<\/a>)<\/li><\/ul><ul><li>Basic project based on Maven &#8211; Use this version of the project to run the MapReduce application locally on your own PC (<span style=\"color:#f80404\" class=\"has-inline-color\"><strong>DO NOT USE IT AT LAIB1<\/strong><\/span>)<ul><li>Import it using Import\/Maven\/Existing Maven Projects<ul><li>Linux, macOS, Windows (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab10_Skeleton.zip\" target=\"_blank\">Lab10_Skeleton.zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><ul><li>Example files \u2013 tweets (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/exampledata_tweets.zip\" target=\"_blank\">exampledata_tweets.zip<\/a>)<\/li><li>Solution<ul><li><a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/Lab10_Sol.zip\" target=\"_blank\" rel=\"noreferrer noopener\">Lab10_sol.zip<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<hr class=\"wp-block-separator has-css-opacity eplus-wrapper\"\/>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<h3 class=\"eplus-wrapper wp-block-heading\" id=\"exam-examples\">Exam examples<\/h3>\n\n\n\n<p class=\" eplus-wrapper\">Pay attention that from the academic year 2020\/21 the exam is <strong><span style=\"color:#f30a0a\" class=\"has-inline-color\">closed book<\/span><\/strong><\/p>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-6bc6a9\"><li>Spark Streaming &#8211; Examples of multiple choice questions (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/ExamplesMultipleChoiceQuestions.pdf\" target=\"_blank\">pdf<\/a>)<ul><li>Answers<ul><li>Question 1: (c)<\/li><li>Question 2: (d)<\/li><li>Question 3: (b)<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-74cd86\"><li>Exam June 30, 2017 <ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2017\/06\/Exam20170630_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/05\/Exam20170630.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 14, 2017 <ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2017\/07\/Exam20170714_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/05\/Exam20170714.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam September 14, 2017 <ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2017\/09\/Exam20170914_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (a)<\/li><li>Question 2: (b)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/05\/Exam20170914.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam June 26, 2018<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/06\/Exam20180626_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (c)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/06\/DraftSolutionv1.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 16, 2018<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/07\/Exam20180716_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (a)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/07\/DraftSolutionv1_20180716.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam September 3, 2018<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2018\/09\/Exam20180903_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/06\/DraftSolutionv1_201809003.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam February 15, 2019<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/03\/Exam20190215_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/06\/DraftSolutionv1_20190215.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 2, 2019<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/07\/Exam20190702_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (a)<\/li><li>Question 2: (b)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/07\/BozzaSoluzionev1_20190702.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 18, 2019<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/07\/Exam20190718_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (b)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2019\/07\/DraftSolutionExam20190718_v1.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 2, 2020<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/07\/BD_Exam20200702.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (a)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/07\/DraftSolutionExam20120702.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 16, 2020<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/07\/BD_Exam20200716.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (b) \u2013 Note that there are two actions and hence the input file is read two times.<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/07\/DraftSolutionExam20120716.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam September 17, 2020<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/09\/BD_Exam20200917.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2020\/09\/DraftSolutionExam20120917.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam February 5, 2021<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2021\/02\/BD_Exam20210205.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2021\/02\/DraftSolutionExam20210205.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam June 30, 2021<ul><li>Exam (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2021\/12\/BD_Exam20210630.pdf\" target=\"_blank\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (a)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/wordpress\/wp-content\/uploads\/2021\/07\/DraftSolutionExam20210630.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-d02e11\"><li>Exam February 2, 2022<ul><li>Exam &#8211; Version #1 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/BD_Exam20220202_v1.pdf\" target=\"_blank\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (b)<\/li><li>Question 2: (d)<\/li><li>Source code\/Eclipse projects (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/Draft20200202.zip\" target=\"_blank\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Exam &#8211; Version #2 (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/BD_Exam20220202_v2.pdf\" target=\"_blank\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (c)<\/li><li>Question 2: (c)<\/li><li>Source code\/Eclipse projects (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/Draft20200202.zip\" target=\"_blank\">zip<\/a>)<\/li><li>Exam February 2, 2022<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<p class=\" eplus-wrapper\"><\/p>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-a24487\"><li>Exam February 21, 2022<ul><li>Exam &#8211; Version #1 (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/BD_Exam20220221_v1.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (b)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/Draft20200221.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><li>Exam &#8211; Version #2 (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/BD_Exam20220221_v2.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (d)<\/li><li>Question 2: (d)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/02\/Draft20200221.zip\">zip<\/a>)<\/li><li>Exam February 21, 2022<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam July 4, 2022<ul><li>Exam (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/07\/BD_Exam20220704.pdf\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (c)<\/li><li>Question 2: (d)<\/li><li>Source code\/Eclipse projects (<a href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/07\/Draft20220704.zip\">zip<\/a>)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li>Exam September 6, 2022<ul><li>Exam (<a rel=\"noreferrer noopener\" href=\"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-content\/uploads\/2022\/09\/BD_Exam20220906.pdf\" target=\"_blank\">pdf<\/a>)<ul><li>Draft of the solution<ul><li>Question 1: (c)<\/li><li>Question 2: (d)<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<hr class=\"wp-block-separator has-css-opacity eplus-wrapper\"\/>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer eplus-wrapper\"><\/div>\n\n\n\n<h3 class=\"eplus-wrapper wp-block-heading\" id=\"additional-material\">Additional material<\/h3>\n\n\n<ul class=\"eplus-wrapper wp-block-list eplus-styles-uid-c0d60e\"><li>Slides and screencasts about Java (kindly provided by prof. Torchiano) (<a href=\"http:\/\/dbdmg.polito.it\/~paolo\/JavaMaterials\/02JEY%20-%20Object%20Oriented%20Programming.html\">link<\/a>)<ul><li>Suggested slides\/lectures for those students who have never used Java<ul><li>OO Paradigm and UML (The UML part is not mandatory)<\/li><li>The Java Environment<\/li><li>Java Basic Features<\/li><li>Java Inheritance<\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n<div class=\"wp-block-buttons eplus-wrapper is-layout-flex wp-block-buttons-is-layout-flex\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>General Information SSD: ING-INF\/05 CFU: 6 Professor: Paolo Garza Teaching Assistant: Luca Colomba Announcements 16-09-21: The first lecture is scheduled for September 27, 2021 at 16:00 in Classroom R1 23-09-21: No lab activities during the first two weeks of the course 27-09-21: I created a page for &#8220;01QYDOV &#8211; Big &hellip;<\/p>\n","protected":false},"author":5,"featured_media":1517,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"editor_plus_copied_stylings":"{}","footnotes":""},"categories":[37],"tags":[],"class_list":["post-1347","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-courses"],"_links":{"self":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/1347","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/comments?post=1347"}],"version-history":[{"count":126,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/1347\/revisions"}],"predecessor-version":[{"id":4536,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/1347\/revisions\/4536"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/media\/1517"}],"wp:attachment":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/media?parent=1347"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/categories?post=1347"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/tags?post=1347"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}