Wikipedia Viewer: Wikipedia API & Cross-Origin Requests (SOLVED)

Banner with text saying "Wikipedia CORS SOLVED"

A lot of people working on the freeCodeCamp Wikipedia Viewer run into problems making requests to the Wikipedia API. They plug the request URL into the search browser and it pulls up the data just fine.

But as soon as they start making the request inside their JavaScript, all sorts of error messages start popping up in the console and no data seems to be getting through.

According to the cross-site request portion of the API documentation, there are two ways to make a cross-site request to a MediaWiki site: JSONP and CORS.

You can bypass these issues by using JSONP instead, but in this article I’m going to be focusing on making the request work using the CORS method.

Wikipedia API Search Example

We’ll start by trying to reverse-engineer the example that freeCodeCamp gives us: a search for the term butterfly that yields a long list of pages containing the word butterfly.

A screenshot of the Free Code Camp Wikipedia Viewer search result page; shows the results upon searching for the term 'butterfly'
The freeCodeCamp Wikipedia Viewer and results for the search term ‘butterfly’

I’m going to use this request URL to get the same sort of results:

https://en.wikipedia.org/w/api.php?action=opensearch&search=butterfly&format=json

You’ll note that I’m using the opensearch action, so let’s look at the OpenSearch action in more detail. The API documentation for OpenSearch gives us this handy blurb about what it does:

Get pages whose name matches a given string. When limit is reached, results are ordered by number of incoming links.

The page gives us more parameters to use to configure our request URL, but we’ll just stick to the basics right now.

You should get something like this if you visit the above link.

Screenshot of the JSON data returned when visiting the request URL in the browser
If the data looks a bit messier than this, don’t panic! I have a JSON Formatter installed through Chrome.

Now for the code:

$.ajax(wikiLink, {
      dataType: "json",
      type: "GET",
      success: function(data) {
        console.log(data);
      }

I have the wikiLink variable set to the request URL shown above, and we’re specifying that we want to GET JSON data back from the API. If the request is successful, the data will be logged to the console for us to look at.

But, this is where we run into a problem…

Wikipedia CORS

Screenshot showing the error message returned in the console, stating that the XHR could not load due to our origin not being allowed access

If we submit the request as-is in our development environment, we get this message from the console:

XMLHttpRequest cannot load https://en.wikipedia.org/w/api.php?action=opensearch&search=butterfly&format=json. No ‘Access-Control-Allow-Origin’ header is present on the requested resource. Origin ‘http://localhost’ is therefore not allowed access.

I’m developing on a local server, so your origin may be different. But what the heck does that mean?

If you go to the API sandbox, you’ll see this blurb under the origin parameter:

When accessing the API using a cross-domain AJAX request (CORS), set this to the originating domain. This must be included in any pre-flight request, and therefore must be part of the request URI (not the POST body)...

For non-authenticated requests, specify the value *. This will cause the Access-Control-Allow-Origin header to be set, but Access-Control-Allow-Credentials will be false and all user-specific data will be restricted.

This is discussed in more detail in the CORS section of the documentation.

Upon reading this part of the documentation, we see that request URL must contain the origin parameter. Since we have a non-authenticated request (i.e. not from one of the allowed domains) we have to specify the origin with an asterisk.

But first let’s take a look at what this is saying in the context of the Chrome Developers console. When you open the Network tab and enter the XHR tab, you’ll see that the request was actually sent a 200 status back.

Screenshot showing the XHR network tab of Developer Tools

Despite being denied access, you can see that the resource was actually transmitted. If you click on the request name, you’ll be able to actually see the requested data under the Response tab. Now let’s look at the Headers tab:

Screenshot showing the Headers section of the Network tab; none of the headers mentioned in the documentation are present
You can see that the Response Headers section is devoid of the Access-Control-Allow-Origin and Access-Control-Allow-Credentials that we keep reading about.

Those CORS headers are what Wikipedia needs to be sending us in order for our JavaScript to be allowed access to the data.

In that case, we can get the API to send us those headers by specifying our origin using the asterisk just as the API documentation noted.

The Origin Solution

Let’s tag &origin=* onto the end of our original request URL, giving us a request URL of:

https://en.wikipedia.org/w/api.php?action=opensearch&search=butterfly&format=json&origin=*

So what happens when we send a request to the API using that URL instead?

Now, we still have the same OK status code but the Access-Control-Allow-Origin and Access-Control-Allow-Credentials have been added to the header.

You can also do this by leaving &origin=* off the URL, and simply adding this origin parameter into the ajax request instead:

$.ajax(wikiLink, {
      dataType: "json",
      data: {
        origin: "*"
      },
      type: "GET",
      success: function(data) {
        console.log(data);
      }

Going back to the console tab of our Developer Tools, we can finally see the data!

The console log showing JSON data received from the Wikipedia API for the search term 'butterfly'

For more information about the Wikipedia API and this CORS solution in particular, check out this helpful answer on StackExchange.

Update: BONUS

This post has gotten more traffic than any of my other web dev posts, and I totally understand why. FreeCodeCamp just throws you into the section on APIs without any prep and holy ****, APIs are ridiculously hard to comprehend when you’re first introduced to them.

Even when I’d completed a couple of the API projects,  I still didn’t really understand what the heck I was doing.

¯\_(ツ)_/¯

One of the main resources that was a lifesaver early in the game when I was about ready to give up was Treehouse’s AJAX Basics class. (FYI I’m a total Treehouse fangirl because they are one of the only e-learning providers that actually speak to me at my level, minus the needless jargon.)

If you’re interested in checking it out, click the link below to go to the class page and check out the material that’s covered; it includes AJAX concepts, retrieving and working with JSON, and much more. You have the option to try a 7-day free trial, so there’s nothing to lose!

treehouse banner

Published by lupe

I'm a web developer at an awesome web design agency in PDX. I'm passionate about learning, playing, tinkering, and blogging about everything from coding to DIY experimentations.

Join the Conversation

28 Comments

  1. Thanks! This really helped me get a deeper understanding of the solution (and learn a new use for Chrome’s dev tools 🙂 )

  2. Hi Lupe,
    Thanks for a great article!!! The CORS section is extremely helpful, I am so glad I came across your blog.

    I do have a question regarding the parameter opensearch. How did you even find this parameter? I read through the API: Main Page, visited a lot of the links, but never came across a mention of opensearch. Before I read your article, I was guessing that in order for the search function to display related articles, I had to use these parameters: action=query&list=categorymembers&format=json&cmtitle=Category:

    Any insight you could offer would be very much appreciated. Thanks again Lupe! And oh, love your cat logo 🙂

    1. Karina, I wish I could remember exactly where I found Open Search in the documentation. I honestly think I just stumbled upon it while frantically opening every page for info on how to get the API working.

      Sorry I can’t be more help! 🙂

  3. I already ran into this problem when working with other API’s — so it was great to see step by step instructions on how to use it here. Also very helpful to see an example of code working with an AJAX call instead of the getJSON shortcut.

  4. Hii, I tried your solution and it’s still a dead end for me..
    My code is running absolutely fine on my local chrome browser but again not on codepen.io
    Here’s my code >

    ——–
    $(function () {
    // click event function
    $(“.icon”).click(function () {
    var input = $(“#search”).val();
    var searchApi = “http://en.wikipedia.org/w/api.php?action=opensearch&format=json&search=” + input ;
    console.log(searchApi);
    $.ajax(searchApi, {
    dataType: “json”,
    data: {
    origin: “*”
    },
    type: “GET”,
    success: function(data) {
    console.log(data);
    //clearing the output for every entery
    $(“.output”).html(” “);
    //appending the search outputs to a ul element
    for (var i = 0; i <= data[1].length; i++) {
    $('.output').prepend("” + data[1][i] + “” + data[2][i] + “”);
    }
    }
    });
    });
    // for enter keypess, the function is called…
    $(“#search”).keyup(function(e){
    if(e.keyCode === 13){
    $(“.icon”).click();
    }
    });
    });
    ———

    1. Try changing the HTTP in your searchAPI variable to HTTPS. It might be a problem of loading mixed (HTTP/HTTPS) content, since CodePen is using an SSL certificate on their site now

      1. Thanks so much Lupe, just that “s” on http, has given me stress for hours. But after adding it, it worked like magic. You are great. Thanks a million.

  5. I’ve literally been searching for hours for a solution to this problem and nothing else has worked except for this.

    Should have just started in the FCC Forum.

    Thanks for writing this!

  6. First of all thanks so much Lupe!

    Now are there any downsides to using origin=* ? Or am I safe to just blindly use it if I need to make a GET request?

    JSONP is discouraged because of the security risks that it imposes. Are there any blaring negatives like that when using origin=*?

    Thank you so much again !

    1. As far as I’m aware, origin=* a specific parameter for the Wikimedia API so it will only work for this specific instance.

      However, I’ve also solved quite a few CORS issues by appending https://cors-anywhere.herokuapp.com/ to the beginning of my request URLs so give that a try!

  7. Thank you so much. I’m surprised that my code worked perfectly 3 weeks ago before I broke off. And when I returned to upload the project, then the issue popped out of no where and i ran nuts.

  8. Thank you so much! Just like that, my cross-origin headache is gone. I imagine scanning crusty API documentation will get easier, but I really needed help and your article was so clear, unassuming and well-explained.
    Again, thank you for helping us all out with understanding this properly.

    1. Absolutely! Learning how to decipher documentation is a mountain to climb in itself, and the documents are usually not written with beginners/novices in mind (unfortunately).

      Thank you for taking the time to respond!

  9. Hola Lupe,
    Thank you for you help me a lot.

    Also, I want to thank you so much for this lines:

    “APIs are ridiculously hard to comprehend when you’re first introduced to them.
    Even when I’d completed a couple of the API projects, I still didn’t really understand what the heck I was doing.”

    A month ago, when I started the freecodecamp front-end course, I didn´t know absolutely nothing about html, css and Javascript, then when I read that API are hard to understand, I do not feel like and idiot anymore.
    I have spent days reading the API documentation trying to figure it out by myself and don´t understand the 90% that I read jajaja. Even in the API Main page at the bottom there are examples of code including JQuery and don´t include the &origin=* and specifies that you should use a POST type request: (https://www.mediawiki.org/wiki/API:Main_page)

    PS: Sorry for my poor english, it´s not my native language.

    Marcelo, and again… THANK YOU!!!!

    1. Hey Marcelo, thanks for leaving a comment! Seriously, don’t feel like an idiot. These are difficult concepts to understand, and every day there’s something new to learn. I feel like us code campers are notoriously hard on ourselves for not immediately grasping the documentation and code samples out there, but it’s like learning another language entirely. Just keep practicing and writing code (and commit to making lots and lots of mistakes!), and one day things will start to click.

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: