apache-spark-benchmark:Apache Spark 框架的测试基准

上传者: 42120563 | 上传时间: 2022-04-15 12:15:20 | 文件大小: 82KB | 文件类型: ZIP
Apache Spark 基准测试 该项目是硕士论文的成果,旨在成为 Apache Spark 框架的测试平台。 其基本思想是能够在软件和硬件的不同环境中运行该框架,以查看其行为,并将获得的结果与类似的解决方案(如 Hive、Redshift 等)进行比较。 这项工作基于伯克利大学 ( ) 进行的大数据基准测试。 本质上,执行的测试是相似的,只是它们是使用 Apache Spark Java API 完成的,并且还测试了一些额外的配置。 此外,该基准测试不仅可以在 Amazon EC2 上运行,还可以在云和本地的任何集群中运行。 项目执行 项目设置 下面列出并简要说明了项目的一般配置参数。 全局配置属性。 这些配置参数是通用的,会影响项目中开发的所有测试。 apache.benchmark.config.global.master :我们想要运行基准的模式。 如果我们想要在本地执行

文件下载

资源详情

[{"title":"( 35 个子文件 82KB ) apache-spark-benchmark:Apache Spark 框架的测试基准","children":[{"title":"apache-spark-benchmark-master","children":[{"title":"pom.xml <span style='color:#111;'> 4.12KB </span>","children":null,"spread":false},{"title":"LICENSE <span style='color:#111;'> 17.60KB </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"resources","children":[{"title":"log4j.properties <span style='color:#111;'> 1009B </span>","children":null,"spread":false},{"title":"benchmark.properties <span style='color:#111;'> 4.99KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"es","children":[{"title":"devcircus","children":[{"title":"apache","children":[{"title":"spark","children":[{"title":"benchmark","children":[{"title":"sql","children":[{"title":"model","children":[{"title":"Document.java <span style='color:#111;'> 1.56KB </span>","children":null,"spread":false},{"title":"Ranking.java <span style='color:#111;'> 2.44KB </span>","children":null,"spread":false},{"title":"UserVisit.java <span style='color:#111;'> 5.10KB </span>","children":null,"spread":false}],"spread":true},{"title":"tests","children":[{"title":"query03","children":[{"title":"Query03ProgrammaticallyTest.java <span style='color:#111;'> 12.74KB </span>","children":null,"spread":false},{"title":"Query03HiveTest.java <span style='color:#111;'> 6.93KB </span>","children":null,"spread":false},{"title":"Query03ReflectionTest.java <span style='color:#111;'> 10.69KB </span>","children":null,"spread":false},{"title":"Query03Test.java <span style='color:#111;'> 4.31KB </span>","children":null,"spread":false}],"spread":false},{"title":"query02","children":[{"title":"Query02HiveTest.java <span style='color:#111;'> 6.60KB </span>","children":null,"spread":false},{"title":"Query02Test.java <span style='color:#111;'> 2.85KB </span>","children":null,"spread":false},{"title":"Query02ProgrammaticallyTest.java <span style='color:#111;'> 10.35KB </span>","children":null,"spread":false},{"title":"Query02ReflectionTest.java <span style='color:#111;'> 9.17KB </span>","children":null,"spread":false}],"spread":false},{"title":"query01","children":[{"title":"Query01ReflectionTest.java <span style='color:#111;'> 8.87KB </span>","children":null,"spread":false},{"title":"Query01Test.java <span style='color:#111;'> 2.49KB </span>","children":null,"spread":false},{"title":"Query01ProgrammaticallyTest.java <span style='color:#111;'> 9.56KB </span>","children":null,"spread":false},{"title":"Query01HiveTest.java <span style='color:#111;'> 6.88KB </span>","children":null,"spread":false}],"spread":false},{"title":"query04","children":[{"title":"Query04Test.java <span style='color:#111;'> 3.54KB </span>","children":null,"spread":false},{"title":"Query04HiveTest.java <span style='color:#111;'> 7.04KB </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":true},{"title":"util","children":[{"title":"config","children":[{"title":"ConfigurationManager.java <span style='color:#111;'> 9.23KB </span>","children":null,"spread":false},{"title":"utils","children":[{"title":"PropertiesUtil.java <span style='color:#111;'> 5.00KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"FileHelper.java <span style='color:#111;'> 4.76KB </span>","children":null,"spread":false},{"title":"runner","children":[{"title":"BenchmarkExecutor.java <span style='color:#111;'> 2.05KB </span>","children":null,"spread":false},{"title":"Runner.java <span style='color:#111;'> 8.06KB </span>","children":null,"spread":false}],"spread":false},{"title":"Launcher.java <span style='color:#111;'> 7.49KB </span>","children":null,"spread":false},{"title":"sql","children":[{"title":"SQLTest.java <span style='color:#111;'> 2.54KB </span>","children":null,"spread":false}],"spread":false},{"title":"FormatHelper.java <span style='color:#111;'> 4.90KB </span>","children":null,"spread":false},{"title":"Test.java <span style='color:#111;'> 2.32KB </span>","children":null,"spread":false}],"spread":true},{"title":"Main.java <span style='color:#111;'> 21.09KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"run.sh <span style='color:#111;'> 182B </span>","children":null,"spread":false},{"title":".gitignore <span style='color:#111;'> 706B </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 10.15KB </span>","children":null,"spread":false},{"title":"scripts","children":[{"title":"url_count.py <span style='color:#111;'> 1.13KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}]

评论信息

  • qq_34884126 :
    执行./run.sh 报错: Error: Could not find or load main class es.devcircus.apache.spark.benchmark.Main
    2021-08-13

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明