nanshihui / Scan-T Star 509 Code Issues Pull requests a new crawler based on python with more function including Network fingerprint search python crawler netfingerprint sybersecurity Updated Aug 14, 2018 C
glouw / andvaranaut Star 118 Code Issues Pull requests A dungeon crawler crawler crawl dungeon Updated Mar 4, 2019 C
ArchiveTeam / wget-lua Star 83 Code Issues Pull requests Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication. crawler scraper downloader spider lua ftp scraping crawling archiving wget crawl zstd crawlers warc webarchiving archiveteam wget-lua Updated Jan 29, 2024 C
JFreegman / toxcrawler Sponsor Star 34 Code Issues Pull requests A Tox DHT network crawler crawler dht tox toxcore dht-network Updated Jan 26, 2024 C
erap320 / CrowLeer Star 27 Code Issues Pull requests Powerful C++ web crawler based on libcurl cli crawler download crawling Updated Jun 1, 2024 C
testomato / minicrawler Star 23 Code Issues Pull requests Multiplexing web client supporting HTTP/2 and WHATWG URL compliant parser written in C c ssl parser crawler cookie icu http2 agpl whatwg multiplexing nghttp2 Updated Mar 21, 2024 C
mithro / fastsvncrawler Star 9 Code Issues Pull requests fast-svn-crawler / fastsvncrawler - A tool for listing SVN repository content export crawler svn subversion import vcs Updated Feb 11, 2018 C
devkot / SearchEngine Star 5 Code Issues Pull requests Distributed Search Engine c http crawler webserver ipc telnet multithreading distributed searchengine namedpipe Updated Jul 6, 2018 C
macos2 / Easy-Crawler2 Star 3 Code Issues Pull requests A crawler programme writen with WebkitGtk and Gtk3 c crawler gtk3 webkit2gtk Updated Jan 11, 2020 C
c0d3Gems / crawlr Star 2 Code Issues Pull requests A web crawler that is tuned for grabbing content from news websites. c http crawler web curl libcurl httprequest Updated Mar 12, 2018 C
xiaoloudongfeng / crawler Star 1 Code Issues Pull requests ?步??爬? redis crawler asynchronous bloom-filter Updated Feb 2, 2018 C
yanym / C-Language-crawler-and-PageRank Star 1 Code Issues Pull requests C language crawler including Bloom Filter and PageRank algorithm. crawler bloom-filter pagerank c-language Updated Nov 30, 2018 C
ozansz / simple-web-downloader Sponsor Star 1 Code Issues Pull requests A simple web page downloader program in C c crawler web curl libcurl Updated Sep 1, 2018 C
JohannHospice / ungoliant Star 1 Code Issues Pull requests Moteur de recherche, capable d’explorer recursivement le contenu d’un ensemble de pages HTML ou de sites Web, de les indexer, et permet a un utilisateur d’effectuer une recherche sur leur contenu. crawler curl tidy Updated Sep 11, 2017 C
ThoAppelsin / bu-registration-scrape Star 0 Code Issues Pull requests This project had been developed to crawl on Bogazici University registration system for its schedules, in C, using the CURL library. c crawler scraper bogazici-university Updated Jan 17, 2018 C
iskunalpal / Website-Downloader Star 0 Code Issues Pull requests A web-crawler to recursively download contents of a given domain c search-engine crawler Updated Feb 14, 2017 C
hassenkassim / easycrawler Star 0 Code Issues Pull requests Easy recursive crawler in C c crawler Updated Jul 12, 2017 C