worldwindjava源码-Analysis-of-Flight-Delay-and-Weather-Dataset:飞行延误和天气数据集

上传者: 38671819 | 上传时间: 2025-03-16 14:07:58 | 文件大小: 2.7MB | 文件类型: ZIP
世界风java源码使用 NoSQL 分析航班延误和天气数据集 团队存储勇士 阿比奈·阿格拉瓦尔 安布吉纳扬 尼提哈拉卡蒂 拉胡尔·夏尔马 介绍 该项目的目标是构建一个应用程序,该应用程序可以从两个不同的海量数据存储中摄取、存储、分析和提取有意义的见解。 这些来源中的第一个来源是 NOAA(国家海洋和大气管理局),它为我们提供了来自世界各地站点网络的每小时天气天气观测。 第二个数据源是 UBTS(美国运输服务局),它为我们提供了航班历史和延误情况。 技术栈 Python Java SQL Hadoop HBase 火花 阿帕奇凤凰 阿帕奇飞艇 Scikit-学习 熊猫 决定技术栈的标准 天气和飞行数据集的大小分别约为 750 GB 和 225 GB。 巨大的数据量促使我们构建一个可扩展的分布式 NoSQL 数据库,例如 HBASE 来存储数据 原始形式的数据集不利于分析,需要大量的预处理。 自定义python脚本用于预处理数据 后预处理,我们需要一个可扩展的分布式流程,可以批量上传到 HBase。 Apache Spark 非常适合这里,因为它具有独特的内存处理能力,可以以非常高的速度处

文件下载

资源详情

[{"title":"( 48 个子文件 2.7MB ) worldwindjava源码-Analysis-of-Flight-Delay-and-Weather-Dataset:飞行延误和天气数据集","children":[{"title":"Analysis-of-Flight-Delay-and-Weather-Dataset-master","children":[{"title":"machine-learning","children":[{"title":"Flight-weather-delay-correlation-data.csv <span style='color:#111;'> 5.34MB </span>","children":null,"spread":false},{"title":"flightPredictor.py <span style='color:#111;'> 5.61KB </span>","children":null,"spread":false},{"title":"preprocess.py <span style='color:#111;'> 817B </span>","children":null,"spread":false}],"spread":true},{"title":"preprocessing-scripts","children":[{"title":"extract-station-info-first-lines.sh <span style='color:#111;'> 446B </span>","children":null,"spread":false},{"title":"extract-us-stations.sh <span style='color:#111;'> 434B </span>","children":null,"spread":false}],"spread":true},{"title":"testData","children":[{"title":"2.csv <span style='color:#111;'> 119.52KB </span>","children":null,"spread":false}],"spread":true},{"title":"sparkInjectionJob","children":[{"title":"StorageWarriors","children":[{"title":"weather.yml <span style='color:#111;'> 1.00KB </span>","children":null,"spread":false},{"title":"pom.xml <span style='color:#111;'> 5.21KB </span>","children":null,"spread":false},{"title":"StorageWarriors.iml <span style='color:#111;'> 20.55KB </span>","children":null,"spread":false},{"title":"tempaustin.yml <span style='color:#111;'> 486B </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"com","children":[{"title":"storagewarriors","children":[{"title":"datapipeline","children":[{"title":"processor","children":[{"title":"DataProcessFactory.java <span style='color:#111;'> 1018B </span>","children":null,"spread":false},{"title":"WeatherDataProcessor.java <span style='color:#111;'> 3.71KB </span>","children":null,"spread":false},{"title":"FlightDataProcessor.java <span style='color:#111;'> 3.58KB </span>","children":null,"spread":false},{"title":"DataProcessor.java <span style='color:#111;'> 844B </span>","children":null,"spread":false}],"spread":false},{"title":"conf","children":[{"title":"SparkConfiguration.java <span style='color:#111;'> 2.15KB </span>","children":null,"spread":false},{"title":"JobConfiguration.java <span style='color:#111;'> 1.18KB </span>","children":null,"spread":false},{"title":"DBConfiguration.java <span style='color:#111;'> 1.74KB </span>","children":null,"spread":false}],"spread":false},{"title":"app","children":[{"title":"HBaseIngestionApp.java <span style='color:#111;'> 2.96KB </span>","children":null,"spread":false}],"spread":false},{"title":"datamodel","children":[{"title":"InputParams.java <span style='color:#111;'> 1.27KB </span>","children":null,"spread":false}],"spread":false},{"title":"args","children":[{"title":"HBaseIngestionArgs.java <span style='color:#111;'> 618B </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"flight.yml <span style='color:#111;'> 1.08KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"StorageWarriors_Project1_Phase2_report.pdf <span style='color:#111;'> 1.85MB </span>","children":null,"spread":false},{"title":"preprocess_abhinay","children":[{"title":"crawl_temp.py <span style='color:#111;'> 3.13KB </span>","children":null,"spread":false},{"title":"temperature_file_links <span style='color:#111;'> 4.28KB </span>","children":null,"spread":false},{"title":"air_clean.py <span style='color:#111;'> 2.11KB </span>","children":null,"spread":false},{"title":"crawl.py <span style='color:#111;'> 4.50KB </span>","children":null,"spread":false},{"title":"weather_stn_geo.py <span style='color:#111;'> 473B </span>","children":null,"spread":false},{"title":"us-stations.txt <span style='color:#111;'> 117.17KB </span>","children":null,"spread":false},{"title":"airports.json <span style='color:#111;'> 1.74MB </span>","children":null,"spread":false},{"title":"airportsUS.csv <span style='color:#111;'> 15.56KB </span>","children":null,"spread":false},{"title":"airport.py <span style='color:#111;'> 546B </span>","children":null,"spread":false},{"title":"readme.txt <span style='color:#111;'> 1.22KB </span>","children":null,"spread":false}],"spread":true},{"title":".gitignore <span style='color:#111;'> 45B </span>","children":null,"spread":false},{"title":"AirportTriangulation","children":[{"title":"pom.xml <span style='color:#111;'> 945B </span>","children":null,"spread":false},{"title":"AirportTriangulation.iml <span style='color:#111;'> 1011B </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"resources","children":[{"title":"airport_stn_mapping.csv <span style='color:#111;'> 11.42KB </span>","children":null,"spread":false},{"title":"out.csv <span style='color:#111;'> 16.54KB </span>","children":null,"spread":false},{"title":"airportsUS.csv <span style='color:#111;'> 15.56KB </span>","children":null,"spread":false},{"title":"weatherTest.csv <span style='color:#111;'> 193B </span>","children":null,"spread":false},{"title":"random.txt <span style='color:#111;'> 449B </span>","children":null,"spread":false},{"title":"fileTest.csv <span style='color:#111;'> 229B </span>","children":null,"spread":false},{"title":"us_stn_geoloc.csv <span style='color:#111;'> 42.46KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"model","children":[{"title":"Flight.java <span style='color:#111;'> 1.02KB </span>","children":null,"spread":false},{"title":"Weather.java <span style='color:#111;'> 485B </span>","children":null,"spread":false}],"spread":false},{"title":"app","children":[{"title":"AirportTriangulationApp.java <span style='color:#111;'> 3.87KB </span>","children":null,"spread":false}],"spread":false},{"title":"args","children":[{"title":"AirportTriangulationAppArgs.java <span style='color:#111;'> 1022B </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":true}],"spread":true}],"spread":true},{"title":"README.md <span style='color:#111;'> 8.84KB </span>","children":null,"spread":false},{"title":"PhoenixNote.json <span style='color:#111;'> 35.25KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明