.net网络爬虫源码 ncrawler

上传者: lanya123 | 上传时间: 2024-01-08 23:35:52 | 文件大小: 4.68MB | 文件类型: ZIP
NCrawler是一款国外的开源网络爬虫软件,遵循LGPL许可协议。其HTML处理使用的是htmlagilitypack开源库,采用xpath的方式处理定位网页元素,十分方便。同时其采用HttpWebRequest异步的方式获取网页,采集效率较高。

文件下载

资源详情

[{"title":"( 263 个子文件 4.68MB ) .net网络爬虫源码 ncrawler","children":[{"title":"Build.bat <span style='color:#111;'> 143B </span>","children":null,"spread":false},{"title":"Build.bat <span style='color:#111;'> 113B </span>","children":null,"spread":false},{"title":"App.Config <span style='color:#111;'> 1019B </span>","children":null,"spread":false},{"title":"app.config <span style='color:#111;'> 1003B </span>","children":null,"spread":false},{"title":"App.config <span style='color:#111;'> 557B </span>","children":null,"spread":false},{"title":"App.Config <span style='color:#111;'> 557B </span>","children":null,"spread":false},{"title":"app.config <span style='color:#111;'> 154B </span>","children":null,"spread":false},{"title":"app.config <span style='color:#111;'> 154B </span>","children":null,"spread":false},{"title":"Options.cs <span style='color:#111;'> 33.03KB </span>","children":null,"spread":false},{"title":"ExtendedHtmlUtility.cs <span style='color:#111;'> 26.64KB </span>","children":null,"spread":false},{"title":"NConsoler.cs <span style='color:#111;'> 24.72KB </span>","children":null,"spread":false},{"title":"NCrawlerEntities.Designer.cs <span style='color:#111;'> 11.08KB </span>","children":null,"spread":false},{"title":"NCrawlerEntities.Designer.cs <span style='color:#111;'> 10.96KB </span>","children":null,"spread":false},{"title":"Crawler.cs <span style='color:#111;'> 10.34KB </span>","children":null,"spread":false},{"title":"AspectExtensions.cs <span style='color:#111;'> 10.33KB </span>","children":null,"spread":false},{"title":"WebDownloaderV2.cs <span style='color:#111;'> 8.74KB </span>","children":null,"spread":false},{"title":"PropertyBag.cs <span style='color:#111;'> 8.32KB </span>","children":null,"spread":false},{"title":"StringPatternGenerator.cs <span style='color:#111;'> 6.92KB </span>","children":null,"spread":false},{"title":"Lazy.cs <span style='color:#111;'> 6.63KB </span>","children":null,"spread":false},{"title":"QueueServiceTest.cs <span style='color:#111;'> 5.89KB </span>","children":null,"spread":false},{"title":"RobotService.cs <span style='color:#111;'> 5.19KB </span>","children":null,"spread":false},{"title":"EsentTableDefinitions.cs <span style='color:#111;'> 5.11KB </span>","children":null,"spread":false},{"title":"HtmlDocumentProcessor.cs <span style='color:#111;'> 4.51KB </span>","children":null,"spread":false},{"title":"Program.cs <span style='color:#111;'> 4.39KB </span>","children":null,"spread":false},{"title":"StreamExtensions.cs <span style='color:#111;'> 4.39KB </span>","children":null,"spread":false},{"title":"ContentCrawlerRules.cs <span style='color:#111;'> 4.36KB </span>","children":null,"spread":false},{"title":"EsentInstance.cs <span style='color:#111;'> 4.32KB </span>","children":null,"spread":false},{"title":"Crawler.Events.cs <span style='color:#111;'> 4.29KB </span>","children":null,"spread":false},{"title":"AspectF.cs <span style='color:#111;'> 4.27KB </span>","children":null,"spread":false},{"title":"HtmlAgilityPackExtensions.cs <span style='color:#111;'> 4.14KB </span>","children":null,"spread":false},{"title":"Resources.Designer.cs <span style='color:#111;'> 3.87KB </span>","children":null,"spread":false},{"title":"IsolatedStorageCrawlerHistoryService.cs <span style='color:#111;'> 3.86KB </span>","children":null,"spread":false},{"title":"EsentCrawlQueueService.cs <span style='color:#111;'> 3.71KB </span>","children":null,"spread":false},{"title":"HistoryServiceTest.cs <span style='color:#111;'> 3.60KB </span>","children":null,"spread":false},{"title":"Resources.Designer.cs <span style='color:#111;'> 3.45KB </span>","children":null,"spread":false},{"title":"Resources.Designer.cs <span style='color:#111;'> 3.45KB </span>","children":null,"spread":false},{"title":"EsentCrawlerHistoryService.cs <span style='color:#111;'> 3.42KB </span>","children":null,"spread":false},{"title":"SimpleCrawlDemo.cs <span style='color:#111;'> 3.36KB </span>","children":null,"spread":false},{"title":"IsolatedStorageCrawlerQueueService.cs <span style='color:#111;'> 3.34KB </span>","children":null,"spread":false},{"title":"Arguments.cs <span style='color:#111;'> 3.12KB </span>","children":null,"spread":false},{"title":"MemoryStreamWithFileBackingStore.cs <span style='color:#111;'> 3.01KB </span>","children":null,"spread":false},{"title":"GoogleLanguageDetection.cs <span style='color:#111;'> 2.94KB </span>","children":null,"spread":false},{"title":"Crawler.Properties.cs <span style='color:#111;'> 2.92KB </span>","children":null,"spread":false},{"title":"CrawlerQueueEntry.cs <span style='color:#111;'> 2.57KB </span>","children":null,"spread":false},{"title":"FileCrawlHistoryService.cs <span style='color:#111;'> 2.56KB </span>","children":null,"spread":false},{"title":"AdvancedCrawlDemo.cs <span style='color:#111;'> 2.56KB </span>","children":null,"spread":false},{"title":"ObjectExtensions.cs <span style='color:#111;'> 2.56KB </span>","children":null,"spread":false},{"title":"NCrawlerModule.cs <span style='color:#111;'> 2.29KB </span>","children":null,"spread":false},{"title":"CrawlStep.cs <span style='color:#111;'> 2.29KB </span>","children":null,"spread":false},{"title":"SitemapProcessor.cs <span style='color:#111;'> 2.22KB </span>","children":null,"spread":false},{"title":"iTextSharpPdfProcessor.cs <span style='color:#111;'> 2.21KB </span>","children":null,"spread":false},{"title":"LinkExtractionProcessor.cs <span style='color:#111;'> 2.12KB </span>","children":null,"spread":false},{"title":"FakeDownloader.cs <span style='color:#111;'> 2.06KB </span>","children":null,"spread":false},{"title":"DictionaryCache.cs <span style='color:#111;'> 2.04KB </span>","children":null,"spread":false},{"title":"IndexerDemo.cs <span style='color:#111;'> 2.01KB </span>","children":null,"spread":false},{"title":"FileCrawlQueueService.cs <span style='color:#111;'> 1.98KB </span>","children":null,"spread":false},{"title":"IFilterProcessor.cs <span style='color:#111;'> 1.92KB </span>","children":null,"spread":false},{"title":"DbCrawlQueueService.cs <span style='color:#111;'> 1.90KB </span>","children":null,"spread":false},{"title":"EMailEntityExtractionProcessor.cs <span style='color:#111;'> 1.81KB </span>","children":null,"spread":false},{"title":"ColorConsoleTraceListener.cs <span style='color:#111;'> 1.76KB </span>","children":null,"spread":false},{"title":"DbCrawlerHistoryService.cs <span style='color:#111;'> 1.73KB </span>","children":null,"spread":false},{"title":"TestModule.cs <span style='color:#111;'> 1.71KB </span>","children":null,"spread":false},{"title":"CrawlerRulesService.cs <span style='color:#111;'> 1.68KB </span>","children":null,"spread":false},{"title":"Program.cs <span style='color:#111;'> 1.63KB </span>","children":null,"spread":false},{"title":"FindBrokenLinksDemo.cs <span style='color:#111;'> 1.52KB </span>","children":null,"spread":false},{"title":"IEnumerableExtensions.cs <span style='color:#111;'> 1.50KB </span>","children":null,"spread":false},{"title":"CrawlUsingIsolatedStorage.cs <span style='color:#111;'> 1.40KB </span>","children":null,"spread":false},{"title":"AssemblyInfo.cs <span style='color:#111;'> 1.39KB </span>","children":null,"spread":false},{"title":"CrawlUsingEsentStorage.cs <span style='color:#111;'> 1.38KB </span>","children":null,"spread":false},{"title":"ThreadSafeCounter.cs <span style='color:#111;'> 1.38KB </span>","children":null,"spread":false},{"title":"CrawlUsingDb4oStorage.cs <span style='color:#111;'> 1.37KB </span>","children":null,"spread":false},{"title":"CrawlUsingDbStorage.cs <span style='color:#111;'> 1.37KB </span>","children":null,"spread":false},{"title":"Db4oHistoryService.cs <span style='color:#111;'> 1.36KB </span>","children":null,"spread":false},{"title":"CrawlUsingSQLiteDbStorage.cs <span style='color:#111;'> 1.36KB </span>","children":null,"spread":false},{"title":"Db4oQueueService.cs <span style='color:#111;'> 1.35KB </span>","children":null,"spread":false},{"title":"Program.cs <span style='color:#111;'> 1.34KB </span>","children":null,"spread":false},{"title":"UrlExtensions.cs <span style='color:#111;'> 1.31KB </span>","children":null,"spread":false},{"title":"HistoryServiceBase.cs <span style='color:#111;'> 1.24KB </span>","children":null,"spread":false},{"title":"NativeTaskRunnerService.cs <span style='color:#111;'> 1.23KB </span>","children":null,"spread":false},{"title":"RequestState.cs <span style='color:#111;'> 1.21KB </span>","children":null,"spread":false},{"title":"AspectExtensions.cs <span style='color:#111;'> 1.19KB </span>","children":null,"spread":false},{"title":"CrawlerQueueServiceBase.cs <span style='color:#111;'> 1.13KB </span>","children":null,"spread":false},{"title":"UriExtensions.cs <span style='color:#111;'> 1.12KB </span>","children":null,"spread":false},{"title":"TempFile.cs <span style='color:#111;'> 1.11KB </span>","children":null,"spread":false},{"title":"MP3FileProcessor.cs <span style='color:#111;'> 1.10KB </span>","children":null,"spread":false},{"title":"FileStorageModule.cs <span style='color:#111;'> 1.08KB </span>","children":null,"spread":false},{"title":"IsolatedStorageModule.cs <span style='color:#111;'> 1.07KB </span>","children":null,"spread":false},{"title":"LambdaFilter.cs <span style='color:#111;'> 1.06KB </span>","children":null,"spread":false},{"title":"SystemTraceLoggerService.cs <span style='color:#111;'> 1.04KB </span>","children":null,"spread":false},{"title":"SharedAssemblyInfo.cs <span style='color:#111;'> 1000B </span>","children":null,"spread":false},{"title":"FileSystemHelpers.cs <span style='color:#111;'> 991B </span>","children":null,"spread":false},{"title":"ThreadTaskRunnerService.cs <span style='color:#111;'> 977B </span>","children":null,"spread":false},{"title":"EsentServicesModule.cs <span style='color:#111;'> 936B </span>","children":null,"spread":false},{"title":"Cursor.cs <span style='color:#111;'> 933B </span>","children":null,"spread":false},{"title":"DbServicesModule.cs <span style='color:#111;'> 921B </span>","children":null,"spread":false},{"title":"Db4oServicesModule.cs <span style='color:#111;'> 918B </span>","children":null,"spread":false},{"title":"IWebDownloader.cs <span style='color:#111;'> 877B </span>","children":null,"spread":false},{"title":"TextDocumentProcessor.cs <span style='color:#111;'> 841B </span>","children":null,"spread":false},{"title":"MaximumCrawlTimeTest.cs <span style='color:#111;'> 812B </span>","children":null,"spread":false},{"title":"DisposableBase.cs <span style='color:#111;'> 741B </span>","children":null,"spread":false},{"title":"......","children":null,"spread":false},{"title":"<span style='color:steelblue;'>文件过多,未全部展示</span>","children":null,"spread":false}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明