A valuable resource to unlock the Power of cURL and SPARQL

With the rise of linked data and the semantic web, knowing how to query SPARQL endpoints using cURL has become an essential skill.

At Cognizone, we are firm believers in the importance of mastering the art of querying SPARQL endpoints using cURL for those who work with linked data and the semantic web.

As a widely supported and user-friendly tool, cURL is a lightweight solution that enables developers and data scientists to efficiently automate the process of extracting data from a plethora of sources.

Whether you are a developer, data scientist, or simply someone who is interested in working with linked data, understanding how to use cURL to query SPARQL endpoints is a valuable asset.

SPARQL, a powerful query language

SPARQL (SPARQL Protocol and RDF Query Language) is a query language for retrieving and manipulating data stored in RDF (Resource Description Framework) a W3C standard for representing information as triples (subject-verb-predicate structure) inside knowledge graphs. It is similar in purpose to SQL (Structured Query Language) which is used for relational databases but is designed specifically for the RDF data model. It allows users to write complex queries to extract specific data from an RDF database.

SPARQL offers flexibility, interoperability, expressiveness, and scalability and is well-suited for Linked Data (data distributed across multiple sources). However, in order to query this Linked Data, it needs to be accessible via a SPARQL endpoint.

A SPARQL Endpoint is one of the points of presence on an HTTP network that is capable of receiving and processing SPARQL protocol requests. The SPARQL Protocol defines how to send SPARQL queries and receive results. The SPARQL Endpoint is identified by a URL that is commonly referred to as a “SPARQL Endpoint URL” and is typically part of an automated SPARQL implementation.

One of the ways to access these SPARQL endpoints is through cURL.

cURL is a computer software project providing a library (libcurl) and a command-line tool (curl) for transferring data using various network protocols and making HTTP requests from the command line. This makes it a great tool for querying SPARQL endpoints, as it allows for easy integration into scripts and automation processes.

The main Benefits of using SPARQL and cURL

The main advantages of using SPARQL are:

• Its flexibility
  Because it is based on RDF, a format that allows for the representation of data in a variety of ways, SPARQL can be used to query a wide range of data sources. This allows for the integration of data from different sources, and the ability to query data in different contexts.
  
• Its interoperability
  SPARQL is a standard language, which means that it can be used to query data from any source that is represented in RDF format. This allows for interoperability between different data sources and applications and makes it easy to share and reuse data.
  
• Its power and expressiveness
  SPARQL has a more complex, graph-based syntax than SQL, which allows for more powerful and expressive querying of RDF data. This makes it possible to perform more complex queries, such as querying data based on patterns or relationships between different pieces of data.
  
• Connect and use Linked Data
  SPARQL is well suited for querying Linked Data to connect and use data across different sources. This enables the ability to discover new insights and knowledge through the connections between different pieces of data.
  
• Enable distributed data storage and processing
  Because SPARQL is based on a graph model, it is well suited for large datasets and can scale to handle large amounts of data.
  

The main advantages of using cURL are:

• Its integration into scripts and automation
  cURL is a command line tool, which means it can be easily integrated into scripts and automation processes.
  
• Custom HTTP
  cURL allows for easy customization of the HTTP request, such as setting headers and adding query parameters.
  
• Its flexibility
  cURL can be used to send both GET and POST requests, which allows for flexibility in the type of query being made.
  
• Interoperability
  cURL can be used to receive the response in a variety of formats, including XML, JSON, and CSV. This can be useful for parsing and processing the query results.
  
• Security
  cURL supports authentication and security by allowing to the passing of the appropriate credentials or certificates. This is particularly useful when querying SPARQL endpoints protected by authentication or SSL.
  

How to make a SPARQL query via cURL?

To make a SPARQL query via cURL, you first need to know the URL of the SPARQL endpoint you want to query. You can then use cURL to send a GET or POST request to this endpoint, along with the SPARQL query you want to make.

For example, to make a SPARQL query for all the books in a bookstore via cURL, the following command can be used:

curl -H "Accept: application/sparql-results+json" -G "<http://example.com/sparql>" --data-urlencode "query=SELECT ?book WHERE { ?book a <http://example.com/ontology/Book> }"

This command sends a GET request to the SPARQL endpoint located at “http://example.com/sparql“, with the query "SELECT ?book WHERE { ?book a http://example.com/ontology/Book }". The -H option is used to set the “Accept” header to “application/sparql-results+json“, indicating that the response should be returned in JSON format.

Now that you have a general idea of how it works, let’s look at our selection of different queries with examples for each type.

---

Forms of queries

We have two types of queries: Read-oriented and Write-oriented queries.

Read-oriented queries are used to retrieve data and are typically taking the form of SELECT or ASK statements, which retrieve or check for the existence of specific data.

Write-oriented queries, on the other hand, are used to modify the data and are taking the form of INSERT, UPDATE, and DELETE statements.

Read oriented queries

the query	Description
SELECT	The Solution Modifier takes the form of a SELECT list, which is very much like a SQL SELECT list, that projects a query solution in tabular form
CONSTRUCT	The Solution Modifier takes the form of an RDF sentence/statement graph specification
DESCRIBE	A variant of CONSTRUCT where the Solution Modifier takes the form of an RDF sentence/statement graph specification that describes one or a selection of entities
ASK	This type of query returns a simple boolean (Yes or No) query solution

Write oriented queries

The query	Description
CREATE	Creates a new empty Named Graph
INSERT	Adds RDF sentences to a Named Graph explicitly or based on conditions satisfied in the SPARQL Query body
INSERT Data	Adds Data based as “fixed” triples
COPY	Copies RDF sentences between Named Graphs
ADD	Adds (or appends) RDF sentences to a Named Graph
MOVE	Move RDF sentences from one Named Graph to another Named Graph; i.e., the Source Named Graph and its sentences are permanently removed after successful recreation in the Target Named Graph
DELETE	Removes RDF sentences that satisfy conditions in the SPARQL Query body from the Target Named Graph. DELETE queries can explicitly target specific statements
DELETE Data	Deletes Data based as “fixed” triples
CLEAR	Removes all RDF sentences in a specific Named Graph i.e., you end up with an empty Named Graph
DROP	Removes a Named Graph in its entirety from the DBMS or Triple/Quad Store

Querying SPARQL endpoint via cURL

Querying SPARQL endpoints via cURL is a straightforward process that involves sending an HTTP request to the endpoint with the SPARQL query as the payload.

To query SPARQL endpoints via Curl, the following steps are to be noted:

1. Accept header needs to be set. In Sparql 1.1 the select queries can return the following Content-types:
   • text/csv
   • application/sparql-results+json
   • application/rdf+xml
     
     
2. Then --data-urlencode is responsible for encoding the query of sparql-query.rq
   
   
3. Select the endpoint that we want to send the http request (For the example below, we have used the production url).
   
   
4. We can save the results in a csv file. If we don’t want to create a file with the results we can remove the last part and request just which will echo the results in our terminal.

`curl -H "Accept: text/csv" --data-urlencode query@sparql-query.rq https://fedlex.data.admin.ch/sparqlendpoint`

Test the Read-oriented queries

I will describe how to test the SELECT, ASK, CONSTRUCT and DESCRIBE queries.

First of all, it is important to make sure that your query is syntactically correct before testing it, otherwise, it may not return the expected results or return an error.

Test a SELECT query

A SELECT query is used to retrieve data. When testing a SELECT query, the goal is to ensure that the query is correctly retrieving the expected data.

This can be done by running the query against a known dataset and comparing the results to the expected results. You can also check that the results do not include any additional data that is not expected.

• Create a test folder;
  
• Open a terminal and execute the command “cd path/of/the/test/folder;`
  

Example of a simple query which will be included inside the sparql-query-consolidations.rq file:

PREFIX jolux: <http://data.legilux.public.lu/resource/ontology/jolux#>
select * {?c a jolux:Consolidation; 
			  jolux:dateApplicability ?startDate; 
			  jolux:dateEndApplicability ?endDate
	 }limit 10

• Execute in the terminal `curl -H "Accept: text/csv" --data-urlencode query@sparql-query-consolidations.rq https://fedlex.data.admin.ch/sparqlendpoint -o result-consolidations.csv`. By setting the accept header as text/csv we expect the result to be displayed in the form of comma-separated values.
  
  
• Additionally, we can try setting the accept header as application/sparql-results+json by executing command `curl -H "Accept: application/sparql-results+json" --data-urlencode query@sparql-query-consolidations.rq https://fedlex.data.admin.ch/sparqlendpoint -o result-consolidations.json` this will return the results back in the form of a json.
  
  
• And by executing `curl -H "Accept: application/rdf+xml" --data-urlencode query@sparql-query-consolidations.rq https://fedlex.data.admin.ch/sparqlendpoint -o result-consolidations.xml` the returned result will be in the form of xml format.

Test an ASK query

An ASK query is used to check the existence of specific data.. When testing an ASK query, the goal is to ensure that the query is correctly returning a boolean value indicating the presence or absence of the specified data.

• Create a new query called sparql-query-consolidation-endDate.rq inside which we will include the following query:

PREFIX jolux: <http://data.legilux.public.lu/resource/ontology/jolux#>
ask where {
<https://fedlex.data.admin.ch/eli/cc/2022/177/20220318> jolux:dateEndApplicability ?endDate.
}

• Execute the following command curl -H "Accept: application/sparql-results+json" --data-urlencode query@sparql-query-consolidation-endDate.rq <https://fedlex.data.admin.ch/sparqlendpoint>
  
  
• The server response of the terminal is in the form of a json.
  
  
• If we change the accept header to text/csv the response will be in the form of a boolean. In this case, we can have two values:
  → “bool” 1 , which is the equivalent of true
  → “bool” 0 , which is the equivalent of false

Test a CONSTRUCT query

A CONSTRUCT query is used to create new RDF triples based on the data in an RDF dataset. When testing a CONSTRUCT query, the goal is to ensure that the query is correctly creating the expected new triples. You can also check that the new triples do not include any errors or unexpected values.

• Create a file called sparql-query-consolidation-construct.rq and include in its content the following query:

construct {
    <https://fedlex.data.admin.ch/eli/cc/2022/177/20220318> ?p ?o.
}
where {
   <https://fedlex.data.admin.ch/eli/cc/2022/177/20220318> ?p ?o.
}

• Execute in the terminal curl -H "Accept: text/csv" --data-urlencode query@sparql-query-consolidation-construct.rq [<https://fedlex.data.admin.ch/sparqlendpoint>](<https://fedlex.data.admin.ch/sparqlendpoint>) where we observe the result in the form of comma-separated values because of the header we set.
  
  
• Other most important content types that can be used are:
  • text/turtle
  • application/rdf+xml
  • application/ld+json
  • application/n-triples

Test a DESCRIBE query

A DESCRIBE query is used to retrieve a description of the resources specified in the query. When testing a DESCRIBE query, the goal is to ensure that the query is correctly retrieving the expected description of the resources.

• Create a file called sparql-query-consolidation-describe.rq and include the following query:

DESCRIBE <https://fedlex.data.admin.ch/eli/cc/2022/177>

• By executing the command *curl -H "Accept: text/csv" --data-urlencode [query@sparql-query-consolidation-describe.rq](<mailto:query@sparql-query-consolidation-describe.rq>)[<https://fedlex.data.admin.ch/sparqlendpoint>](<https://fedlex.data.admin.ch/sparqlendpoint>)* where we observe the result in the form of comma-separated values because of the header we set.
  
• For DESCRIBE we can also use the same content types as we use for CONSTRUCT as both of these queries return data in the form of RDF triples.

---

cURL is an incredibly powerful tool for querying SPARQL endpoints, providing users with the flexibility to send GET and POST requests, allowing it to retrieve responses in different formats. This makes it an ideal choice for extracting RDF data stored in knowledge graphs.

At Cognizone, we understand the full potential of this powerful combination of cURL and SPARQL. Together, they offer flexibility, interoperability, expressiveness, and the ability to connect and use Linked Data.

We strive to share our knowledge and skills, and this is a nice opportunity for us to share with you our expertise on this powerful tool and its role in the Linked Data and Semantic Web space.

---

We appreciate the time you took out of your day to read this article and we hope you found it informative!

You can find more articles on our blog and while you’re there take a look at our resources and insights about building an effective Enterprise Knowledge Graph.

For news and insights find us on Twitter @cognizone

Interested in training a SPARQL endpoint? Contact us at info@cogni.zone

We are a specialist consultancy with long experience in designing and implementing Enterprise Knowledge Graphs in government and other data-intensive sectors. Through our combination of technical skills, industry practitioners, and expertise in open and linked data, we can help you see data challenges from a new perspective.