go语言单并发版爬虫--crawler-v1-v4

上传者: u011127242 | 上传时间: 2021-02-27 17:07:03 | 文件大小: 189KB | 文件类型: ZIP
go语言单并发版爬虫--crawler-v1-v4 crawler-v1-v4 资源中包含一个完成的go语言并发爬虫案例,其中v1为goroutine+多worker,v2为request+worker双队列,v3将其主模块优化为同时支持v1和v2,v4继续新增多城市访问+排除重复的信息; 该案例非常适合作为go语言入门的练手案例; 更多详细信息参考笔者博文: [golang笔记16--go语言并发版爬虫](https://blog.csdn.net/u011127242/article/details/114161224)

文件下载

资源详情

[{"title":"( 47 个子文件 189KB ) go语言单并发版爬虫--crawler-v1-v4","children":[{"title":"crawler-v1-v4","children":[{"title":"crawler-v2","children":[{"title":"fetcher","children":[{"title":"fetcher.go <span style='color:#111;'> 1.14KB </span>","children":null,"spread":false}],"spread":true},{"title":"zhenai","children":[{"title":"parser","children":[{"title":"city.go <span style='color:#111;'> 3.77KB </span>","children":null,"spread":false},{"title":"citylist.go <span style='color:#111;'> 685B </span>","children":null,"spread":false},{"title":"citylist_test.go <span style='color:#111;'> 1.29KB </span>","children":null,"spread":false},{"title":"citylist_test_data.html <span style='color:#111;'> 203.75KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"mian.go <span style='color:#111;'> 575B </span>","children":null,"spread":false},{"title":"scheduler","children":[{"title":"queued.go <span style='color:#111;'> 1.04KB </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 347B </span>","children":null,"spread":false}],"spread":true},{"title":"engine","children":[{"title":"types.go <span style='color:#111;'> 231B </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 544B </span>","children":null,"spread":false},{"title":"worker.go <span style='color:#111;'> 326B </span>","children":null,"spread":false},{"title":"concurrent.go <span style='color:#111;'> 965B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"crawler-v3","children":[{"title":"fetcher","children":[{"title":"fetcher.go <span style='color:#111;'> 1.14KB </span>","children":null,"spread":false}],"spread":true},{"title":"zhenai","children":[{"title":"parser","children":[{"title":"city.go <span style='color:#111;'> 3.77KB </span>","children":null,"spread":false},{"title":"citylist.go <span style='color:#111;'> 685B </span>","children":null,"spread":false},{"title":"citylist_test.go <span style='color:#111;'> 1.29KB </span>","children":null,"spread":false},{"title":"citylist_test_data.html <span style='color:#111;'> 203.75KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"mian.go <span style='color:#111;'> 575B </span>","children":null,"spread":false},{"title":"scheduler","children":[{"title":"queued.go <span style='color:#111;'> 1.04KB </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 476B </span>","children":null,"spread":false}],"spread":true},{"title":"engine","children":[{"title":"types.go <span style='color:#111;'> 231B </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 544B </span>","children":null,"spread":false},{"title":"worker.go <span style='color:#111;'> 326B </span>","children":null,"spread":false},{"title":"concurrent.go <span style='color:#111;'> 1.00KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"crawler-v4","children":[{"title":"fetcher","children":[{"title":"fetcher.go <span style='color:#111;'> 1.14KB </span>","children":null,"spread":false}],"spread":true},{"title":"zhenai","children":[{"title":"parser","children":[{"title":"city.go <span style='color:#111;'> 3.68KB </span>","children":null,"spread":false},{"title":"citylist.go <span style='color:#111;'> 685B </span>","children":null,"spread":false},{"title":"citylist_test.go <span style='color:#111;'> 1.29KB </span>","children":null,"spread":false},{"title":"citylist_test_data.html <span style='color:#111;'> 203.75KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"mian.go <span style='color:#111;'> 575B </span>","children":null,"spread":false},{"title":"scheduler","children":[{"title":"queued.go <span style='color:#111;'> 1.04KB </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 476B </span>","children":null,"spread":false}],"spread":true},{"title":"engine","children":[{"title":"types.go <span style='color:#111;'> 231B </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 544B </span>","children":null,"spread":false},{"title":"worker.go <span style='color:#111;'> 326B </span>","children":null,"spread":false},{"title":"concurrent.go <span style='color:#111;'> 1.36KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"crawler-v1","children":[{"title":"fetcher","children":[{"title":"fetcher.go <span style='color:#111;'> 1.14KB </span>","children":null,"spread":false}],"spread":true},{"title":"zhenai","children":[{"title":"parser","children":[{"title":"city.go <span style='color:#111;'> 3.77KB </span>","children":null,"spread":false},{"title":"citylist.go <span style='color:#111;'> 685B </span>","children":null,"spread":false},{"title":"citylist_test.go <span style='color:#111;'> 1.29KB </span>","children":null,"spread":false},{"title":"citylist_test_data.html <span style='color:#111;'> 203.75KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"mian.go <span style='color:#111;'> 415B </span>","children":null,"spread":false},{"title":"scheduler","children":[{"title":"simple.go <span style='color:#111;'> 347B </span>","children":null,"spread":false}],"spread":true},{"title":"engine","children":[{"title":"types.go <span style='color:#111;'> 231B </span>","children":null,"spread":false},{"title":"simple.go <span style='color:#111;'> 544B </span>","children":null,"spread":false},{"title":"worker.go <span style='color:#111;'> 326B </span>","children":null,"spread":false},{"title":"concurrent.go <span style='color:#111;'> 895B </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明