How to get Trending GitHub Repositories Using Node.js ?
Approach:
- Fetch the entire HTML page and store it as string using request package.
- Load HTML into cheerio and find the CSS selectors to extract repositories details.
Using request package:
request package: The request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default.
Installing request package:
$ npm install request
Note: Run this command in current folder, where ‘app.js’ file located.
Syntax for request:
request(url, (error, response, body) => { if(!error && response.statusCode == 200) { statements to be executed. } }
Parameters:
- url: API endpoint to which request is made.
- response: HTTP response status codes indicate whether a specific HTTP request has been successfully completed.
- body: Response data.
Using cheerio package:
cheerio package: Fast, flexible & lean implementation of core jQuery designed specifically for the server.
Installing cheerio package:
$ npm install cheerio
Note: Run this command in current folder, where ‘app.js’ file located.
Syntax for cheerio:
const cheerio = require('cheerio') const $ = cheerio.load(HTMLString) $(CSS Selector).text('Hello there!') $.html()
// Import request package const request = require( 'request' ); // Import cheerio package const cheerio = require( 'cheerio' ); // Github Trending Page URL const url = 'https://github.com/trending' ; // Get request to the URL request.get(url, (error, response, body) => { // If the response code is 200 and // there is no error if (!error && response.statusCode == 200) { // Load HTML string into cheerio const $ = cheerio.load(body); // Below are the CSS selectors to // fetch the data required let temp = $( '.Box-row' ) let repos = $( '.h3.lh-condensed a' ); let data = []; for (let i = 0; i < repos.length; i++) { let reponame = $(temp[i]) .find( '.h3.lh-condensed a' ) .text().replace( /[\n\r]+|[\s]{2, }/g, ' ' ).trim(); let repolanguage = $(temp[i]) .find( '.f6.text-gray.mt-2 span span[itemprop="programmingLanguage"]' ) .text() .replace(/[\n\r]+|[\s]{2, }/g, ' ' ).trim(); let repostars = $(temp[i]) .find( '.f6.text-gray.mt-2 .muted-link.d-inline-block.mr-3' ) .text() .replace(/[\n\r]+|[\s]{2, }/g, ' ' ).trim(); // Push the fetched data into an object data.push({ 'Repository' : reponame, 'Language' : repolanguage, 'Stars/Forks' : repostars }) } // Display the Object created using console.table console.table(data) } else { console.log( "Unable to fetch data from github" ); } }); |
Output: