Open Data Licencing & Search Engines
Search engines have become man’s new best friend. We rely on them for various purposes, ranging from social media engagement to work-related research. They are vital to our economy, and can play a major role in daily decisions on where to dine or shop.
Our best buddy can tell us where there is work, which qualifications we need to get our dream job, or which province is cheapest to live in (hint: it’s not BC). They have conditioned us to expect reliable and usable information in a matter of seconds.
We are bound to ask though, do search engines primarily exist to answer our questions, or is there another purpose? How many times have you looked online only to find results that did not correspond to what you were looking for?
It could be that you used the wrong keywords. Or is it possible the search engine has an agenda of its own?
This is what Bob Lytle, founder of rel8ed.to Analytics, may have uncovered in a recent research project on local search engine effectiveness. His findings will be presented at a gathering of industry and academic leaders at the 25th International World Wide Web Conference in Montreal April 11 – 15.
The findings hint that local search engine capabilities, provided by companies such as Google, Bing, and regional providers, may not be solely interested in providing end users with useful results. These providers profit from the search transaction whether or not the results are related to our searches.
As an example of the research, tow trucks were searched on both local and global search engines in the City of Toronto. It was observed that only about half of the top results returned were useful.
Simply put, only one in two results led to a company that was reachable and offered tow truck services! Of those top results, only 1 in 3 had a proper licence from the municipal authority, and 1 in 6 businesses advertised were closed. Let me represent these numbers for you:
Does that feel familiar?
It gets worse: in the top results, approximately three out of six companies actually provided usable tow truck services.
The surprise? The lower we scroll, the more likely we are to find a usable tow truck company link!
In the remaining entries, roughly 2/3 of the companies were rated usable.
In other words, the research seems to demonstrate that the lower you go in search results, the more useful the results will be. Google made its name in the search industry with high-quality results.
What’s going on here?
Checking the Licence
The rel8ed.to research was originally conducted to assess the impact of using Open Data licencing information to enhance local search engines. Information from the City of Toronto Municipal Licencing dataset was compared against the results from four major providers to gauge the current status of licencing data in the search engine databases.
The limited licence coverage was not surprising, but the poor overall usefulness of the results came as a shock to Lytle and the research team: “I expected to find some areas for general improvement that would enhance our messaging to government bodies on the benefits of Open Data,” he stated.
“Instead, we found such a broad gap between search results and true company capability that I believe a new market for Open Data may emerge in the search engine space.”
In an emergency situation such as the one below, the user might not care whether the company is actually licenced or not, and would probably not bother verifying if the company selected had a valid towing licence. (In the analysis set, 1/3 of towing companies with a licence entry possessed an invalid or expired certification.)
On the other hand, a licence check might be worthwhile considering that licenced companies do need to be inspected and conform to a certain standard to keep their licences.
This usually means that they are much more likely to offer professional service. Following an off-road accident, to have someone that cares about us and our car may actually avoid further drama!
I mean… how comfortable would you feel if this was your rescue?
So, what’s really going on here?
With so much at stake for local search, why do we get so many seemingly irrelevant results? The answer may lie partly in the internet’s commercial ecosystem.
End-users are not the only players involved on the web. Others include the search engine provider, Search Engine Optimization (SEO) consultants, and the businesses appearing within search results.
The intricate system that links them all is based on a balance between advertisement and end-user experience.
Search engines use a complex combination of webpage text and hidden information to rank the results according to certain criteria. This structure is both efficient and systematic, and allows SEO consultants to observe and deduce the important ranking features of search engines. They use that knowledge to improve the ranking of their clients’ sites.
Those clients are typically businesses that try to advertise their products and will be the income source of both SEOs and search providers.
The search providers benefit both from the businesses and the end-users: their goal is to optimize their database and their search to both advertise businesses, which brings in their income, and display results useful enough for the end user to choose their search engine over others.
rel8ed.to’s research identified three key components making those results useful to us: the business must exist, it must provide the service we need and it must not have an invalid licence. Fair enough.
The problem seems to be that, in the end, if companies and the SEO partners target the ranking system properly, the search engine may display results because they were best fitted for the ranking algorithm, even if that rank may be of little use to the end user.
In short, the inability of the online search engine to validate business operation and credentials leaves consumers open to poor offline experiences.
If we could somehow improve the re-ranking to favour consumer experience, it could involve some serious impacts on the other stakeholders. The process might introduce significant changes to the search position of companies: will the other stakeholders welcome this change? Do they really want the problem to be solved?
Vee Popat, Founder of Ontario Digital Marketing and SEO agency VPDM Digital , predicts that the change would be well received. He stated that SEOs do have moral obligations toward their clients, and that they should use “white hat” strategies to increase the ranking. This is usually done by creating specific marketing personas, developing keywords for those personas and completing a competitive and keyword analysis.
A system of search engine validation of the business operation itself would ensure quality results for quality service. He added that such a system would create a level field for all businesses, aiding SEOs in providing better services for their clients and creating a better user experience.
A possible solution?
A novel approach to solving this problem through Open Data sources is the subject of Lytle’s presentation at the Open Data for Local Search workshop at the www2016 conference. Conference proceedings and the full research paper will be published and discussed in a future post.
In the meantime: don’t panic if you can’t find what you are looking for immediately… just scroll down a little lower than you normally would!
Categorised in: Business Data Analysis, Open Data, Predictive Analytics
This post was written by Valérie Plante-Brisebois