CoolQLCool (CQC) is an open source GraphQL server that allows you to turn websites into GraphQL APIs. It's pretty tubular (I'm incredibly biased tho).
You can play around with it in GraphiQL. Or take a gander at the source on GitHub.
In order of how radical I think they are.
By default CQC will load just the HTML like a normal AJAX request, but if you pass the `wait` or `waitForSelector` arguments in your initial `site` search, CQC will wait a designated time, or for a CSS selector on the page to show up.
In past renditions of web scrapers they've only been able to scrape a single site at once (e.g. Jam API). With CQC you can query multiple sites in a single request by aliasing each `site` call.
With past scrapers it was incredibly difficult to add new fields to query elements by or to grab off of elements. With CQC it will be incredibly easy to have contributors add new fields and functionality.
Some possible features that I'd like to see CQC have some day
curl -X GET -H "Content-Type: application/json" -d '{"query": "{ site(url: \"https://news.ycombinator.com\") { titles: selectAll(elem: \"tr.athing\") { id: attr(name: \"id\") numberOfLinks: count(elem: \".storylink\") link: selectAll(elem: \".storylink\") { text href class classList } } } }"}' https://coolql.cool/graphql
var xhr = new XMLHttpRequest();
xhr.responseType = 'json';
xhr.open('POST', 'https://coolql.cool/graphql');
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.setRequestHeader('Accept', 'application/json');
xhr.onload = function () {
console.log('data returned:', xhr.response);
}
xhr.send(JSON.stringify({ query: `{
site(url: "https://news.ycombinator.com") {
titles: selectAll(elem: "tr.athing") {
id: attr(name: "id")
numberOfLinks: count(elem: ".storylink")
link: selectAll(elem: ".storylink") {
text
href
class
classList
}
}
}
}` }));
require 'net/http'
require 'uri'
require 'json'
uri = URI.parse("https://coolql.cool/graphql")
header = {'Content-Type': 'application/json'}
query = { query: '{
site(url: "https://news.ycombinator.com") {
titles: selectAll(elem: "tr.athing") {
id: attr(name: "id")
numberOfLinks: count(elem: ".storylink")
link: selectAll(elem: ".storylink") {
text
href
class
classList
}
}
}
}' }
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri, header)
request.body = query.to_json
response = http.request(request)
Made by Gavin, deployed on Now