UK Analytical Datasets based on OpenData - A Overview of GeoLytix GeoData

Courtesy of Directions Magazine

UK retailer, Sainsbury, with over 30 analysts, synthesizes data for demographic analyses, location planning and other applications. In this sponsored article, Annette Dellevoet, a senior analyst with Sainsbury’s Network Planning team with responsibility for assessing the viability of new sites and store investments as well as overall estate strategy, provides this review of the GeoLytix data solutions.

Background

We at Sainsbury’s have been a customer of GeoLytix’s consulting services for over a year, and I am passionate about data, especially open data. Therefore I was excited when Blair Freebairn, the Managing Director of GeoLytix, told me this spring they were building a comprehensive set of Geodata for the UK using only OpenData as a source. After reviewing the early output, and comparing them against the data we currently license, we quickly decided to sign up and eagerly awaited delivery of the data.
 
Overview
In total there are over 100 map layers arranged into seventeen ‘themes’ or bundles. These bundles cover everything from a very detailed road display layer to the UK’s administrative geographies and a whole slew of Points of Interest and some innovative derived datasets. A summary of the offer is on the GeoLytix website
 
Licensing
In Britain a very rich set of government data, including nearly all the Ordnance Survey mid-scale vector products, have been released under a wonderfully permissive Open license. The GeoLytix data license, although paid-for, is similarly permissive. This means I don’t have to worry about how many of my users have the data on their machines, or whether I am allowed to share it with colleagues elsewhere in the business or our other partners like Aimia who look after our Nectar card data. As long as the data is being used for Sainsbury’s own purposes I can share it internally and we can use it as we wish.
 
GeoLytix also decided to make some of their datasets truly OpenData, with charges only levied for updates and add-ons . To date these include postcode sector boundaries, Retail Places and a standard set of OA demographics derived from the latest 2011 Census.   
One of the best and most useful datasets is their detailed set of postal sector boundaries for the whole of Great Britain. As a current client, I was given early access to samples of these data as the methodology for creating the boundaries developed and had the chance to give feedback. GeoLytix have built the geographies using an innovative ‘snapping’ algorithm that makes nearly all the boundaries follow real world features like railways, rivers or roads. This results in very intuitive and accurate boundaries. I think it is great that such a key dataset, which was previously only available under restrictive and expensive terms, has been made OpenData. It means many of the smaller GIS teams, and even non GIS people, can now get access to these boundaries. 
 

The GeoLytix Postal Sector Boundaries showing correct postcode point enclosure with real world features as boundaries (click for larger image).

 
Content
Within the main data bundles there are latest versions of all of the existing OS OpenData, but GeoLytix have consolidated all the data, styled it, made it all seamless and corrected lots of little errors. We could have done most of this processing ourselves but it would have taken ages. In addition GeoLytix have ‘mined’ the whole open data portfolio from data.gov.uk and other sources to add in a whole raft of extra key data. Most importantly they have created some new, innovative and very useful derived datasets. One dataset I particularly like, and use often, is the town and suburb boundaries. This collection of polygons represents real ‘places’ that our location planners (and indeed non-GIS specialists) are instantly comfortable with. This data is simply not available from any of other suppliers. We have found many uses for these in particular in building network plans for our Convenience portfolio and in accurately describing the locations of new competitor developments to some of our non-GIS users.
 
Another bundle we get a lot of use out of is the public transport pack. This is based on definitive national transport data. For example it includes over 380,000 bus stops. What is particularly powerful is that GeoLytix have added in passenger volumes, either definitive real values or modelled where they aren’t collected. This is a dataset I have built myself in the past, but it was a real pain trying to source all the data from National Rail, TfL and the smaller transport operators, not to mention matching the volumes up to the correct station locations, so it is great that now GeoLytix has done all the hard work!
 
We also use Retail Places: a set of polygons defining all of the UK’s town centres, retail parks, shopping centres and parades. For the largest town centres there is a 10m grid overlaid that scores the ‘pitch’ within each centre which is a nice extra. Some of the naming of these centres was a bit confusing, and the polygons aren’t as accurate as I would have liked. When I raised my concerns with Blair he took them on board, and produced a new much better release in January 2013. The dataset has been refined to remove the least significant areas and the polygons have been constrained using land use coding from OS Mastermap so they follow the exact lines of the retail areas.  I still have some concerns regarding the naming as names are only unique within towns, making it tricky to find the right location when searching at a national or regional level, also a ranking of places would be really useful, but I feel sure  these issues will be addressed  in the next release. One of the USP’s of GeoLytix’s service is their willingness to accept constructive criticism and work with the customer to make each product the best that it can be.
 

A snapshot of GeoLytix Retail Places and their pitch rating in Nottingham and surrounding area (click for larger image).

 

The GeoLytix Retail Places outlines within Central London. The full dataset is in green; and the Open data is labelled and shown in Red. The Open Retail Places cover 342 of the largest towns and retail parks (click for larger image).

 
GeoLytix’s latest release is a dataset created from the 2011 Census bulk outputs which includes a subset of the most useful demographic counts at OA level in CSV format. It would be very time-consuming to produce this in-house from the many Key Stats and Quick Stats tables released by ONS and NISRA, not to mention working out the differences between the definitions of certain counts for each country, but fortunately GeoLytix has again done all the grafting. Not only that, they have released this as OpenData and included a comprehensive user guide, lookup tables and estimated the proportions of 2001 OA’s in each 2011 OA so that accurate estimates of 2001 Census statistics can be made for the 2011 OA’s for comparison with the latest data. I am looking forward to this dataset being available for the whole of the UK once Scottish Census data becomes available.
 
There are also three collections of Points of Interest (POI). The education one is especially useful to us as it includes all the schools, colleges and universities together with pupil numbers and age ranges and official lookup codes. Education facilities are a key source of non-residential trade for our convenience stores. The sports stadia one that includes capacities, teams etc and covers every location where professional sport is played has some use to us, but in truth, is probably of most interest to my male colleagues for purely personal reasons! Some of the POI layers are of little us to us - including courts and prisons! But I do see how they will be of more interest to other potential users. Again they all come with official identifiers so users should be able to easily link them to other data sources.
 
GeoLytix are currently working with our team to trial building a bespoke demand surface that takes into account multi-channel behaviour and a customer’s mission type.  This will help us to better understand the potential for convenience, superstore and online sales in an area. 
 
The routable road network for use in drive time software is one bundle we do not use. We license a proprietary road network that we have built our own speed-model for using real world speeds. Changing the road network would trigger a re-build of many of our tools and models. 
 
Technical
The data is supplied as either shapefile or MapInfo tab files in British National Grid, this is fine for our purposes and if colleagues need it in other formats or projections I can do that myself very simply in our GIS. Because the data bundles include some very detailed 10K Raster layers and the nominal scale of the vector data is generally 1:10,000 the package is quite hefty at 15 GB. Every data bundle has full documentation explaining the sources and methodology outlines. They also include a comprehensive data dictionary that explains every data field in every layer. The guides and other supporting documents are well written and easy to understand, but there are a few minor errors I have picked up that will hopefully get corrected as the product matures. The data has been designed to be used together so all the layers ‘nest’ perfectly, and the vector layers exactly match the appropriate Raster backgrounds. GeoLytix have included Northern Irish data where possible, which is a big advantage over the regular OS OpenData. Wherever official codes exist they have been included, such as administrative area codes, hospital identifiers and public transport codes etc. This allows me to easily create links from the GeoLytix data to our internally built and external supplied datasets.
 
Commercial
I like the easy and transparent pricing approach from GeoLytix. You can pick three bundles for £8,000 a year, six bundles for £14,000 or have all seventeen for £25,000 (if you want to you can also choose to license each individually). Currently we use the six bundle option, but have signed up to the full pack for next year.  GeoLytix are also more than happy to share any dataset with a potential customer on a 30 day evaluation license, this is much better than only being able to review a specific town or area.
 
The postal geography updates we license only cost £1,200 a year which is about an 80% saving over what we paid before. We have also been able to replace several data sets we previously licensed piece-meal. We have ended up with much wider, richer and up-to-date collection of data than before at less than we used to pay.
 
Summary
As a large team with over thirty analysts and planners we particularly like the flexible unlimited licensing. For smaller teams I think the pricing will be pretty much on par with their current arrangements but the data does include several innovations you won’t see anywhere else and will definitely work out more cost-effective than trying to process all the data yourself. For one-person GIS teams or non-specialists the pricing may put some people off, maybe GeoLytix could introduce more flexible pricing/offers for these smaller users? To get the most out of the data you will need to know your way around your GIS. The data is ‘flat’ and does not come with any reporting or analysis tools. Currently it will not be a huge amount of use to anyone without access to a GIS. 
 
During the nine months we have been using the data I have found GeoLytix to be very responsive and attuned to our needs. When we first licensed the data they came into the office to run the team through all the datasets, and having access to the team who actually build and maintain the data means we can always get an instant, direct and definitive answer to any queries.
Written by Default at 12:00

Location Intelligence Finds it's Way into BI Processes

Courtesy of B-eye Network

Business intelligence (BI) systems have long struggled with the location dimension of the data contained in everything from spreadsheets to data warehouses and the transactional business systems that feed them. 

Customer, product, or channel dimensions are visualized and analyzed with a wide variety of graphical tools. But geographic data is forced into these tools like a square peg in a round hole. 

Help is on the way. Location intelligence brings a new set of tools to the table, ones better suited to the job of tapping into geographic information meaningfully and effectively. 

Location intelligence has a role to play in these BI functions:

Reporting and Visualization
Traditional business intelligence is dominated by standardized reporting with some limited filtering available to business users. Location intelligence plays a similar role by providing maps that give users the ability to pan, zoom, drill up or down, turn geographic layers on or off, and do some limited filtering for thematic mapping.

Within all leading BI front-end tools, interactive maps are replacing or augmenting standard table and chart views of geographic data. Business data is typically mapped as dots or icons, shaded polygons, or "hot spots" based upon some characteristic of the data, such as sales volume or cost. Recently more advanced geographic visualizations such as three-dimensional representations and space-time animations have found their way into some BI tools and end-user dashboards. 

Whether simple or advanced, visualizing data on maps is often a vast improvement over pie and bar charts, in many cases, it leads business analysts and other users to findings and conclusions in much less time and with less effort than spinning , slicing and dicing a cube of tabular information looking for spatial relationships – relationships they might never even find using the old method.
Data Integration and Quality
Geographic data shares many of the same problems associated with other data, such as multiple source systems and spatial data formats, data quality issues and semantic inconsistencies. But it adds a few more, like different geographic reference systems (so things don't line up on the map) or different spatial dimensions (for example, one system maps buildings as points, another as polygons). 

Location intelligence technology can help with spatial data integration and quality improvements. Tools are available to convert various spatial data formats; re-project data from one coordinate system to another; and clean up and validate spatial data before it's used. The goal is to create a single version of the truth for spatial data that can be used throughout an organization to create accurate, meaningful and consistent maps and serve as a foundation for advanced spatial analysis. In the new world of self-service reporting tools for end users, effective processes for spatial data integration and quality management are critical.
Advanced Spatial Analysis
In both business intelligence and location intelligence, a relatively small group of people play with the data to discover what they don't know about it, employing heavier-duty analysis methods such as ad hoc querying or data mining. Somewhere, there is an imaginary line that divides simple location intelligence from advanced spatial analysis. One way to think about the line is to say that all analysis that can be done in the average person's head through visualization is on the simple side, while analytical tasks that require  spatial statistics, clustering and forecasting or other spatial computations and models falls on the advanced side. 

In short, if you can’t get it from looking at the map, you need some advanced techniques. Advanced spatial analysis tools relate to familiar advanced analytics software for statistical analysis, data mining, real-time forecasting and business optimization, but they're modified to address the unique characteristics of spatial data and relationships.
Collaboration
Social networks are driving a rush toward increased support for collaborative BI capabilities. Standardized reporting, querying and advanced analytics all have their limits, and it's often difficult to uncover information about the cause and impact of business problems and corrective actions that could be taken. But often, multiple users working collaboratively can fill in data gaps, raise new considerations and make collective judgments and decisions.

Maps have always been magnets for people, as evidenced by the countless social media sites that use Google Maps, and many business professionals want the same kind of collaborative experience to be available in the workplace. These "prosumers" have been heard, and maps now can easily be created, shared, annotated, extended and re-shared using cloud offerings without the need for IT resources or in-depth location intelligence skills. Self-service has come to location intelligence along with collaboration.

It's easy to see the parallels between business intelligence and location intelligence since  they share major areas of focus and serve similar purposes. But until now they have evolved independently of each other. Today, a perfect storm of technology, data and a DIY mentality is bringing these two ships together and creating a wave of intelligent maps and location-based analytics.

Written by Default at 15:00

Recent Catastrophes Highlight Need for Location Intelligence

Courtesy of Property Casualty 360

Recent catastrophes have highlighted lingering problems for property & casualty insurers: Do carriers understand what their risks are and can they locate their policies and assets. In other words: location intelligence.

Pitney Bowes Software has rolled out Spectrum Spatial, which is part of the Spectrum SOA platform, according to Lamont Norman, global geocoding and risk data product manager for Pitney Bowes Software.

“This allows customers to geocode and perform at-risk quality and location-quality operations in the back office quickly without a software engineer and then be able to pass that information and quickly analyze and understand what the risk is,” says Norman. “We tie that to static risks or what we call slower-developing risks, such as earthquake or hurricane risks.”  

Norman explains the system uses weather feed information about severe storms coming through an area to determine the insurer’s exposures and how many policies can be affected by such a storm.

“We can interface with the customer’s existing systems through web services and pull that information together to give insurers a timely understanding of what is happening when there is a catastrophic event,” he says. “You can operationally explain these are the policies you are interested in and of a certain value, and are part of a particular book of business in the region.”

Companies are working with Pitney Bowes Software on the operational side and as they become more aware of what the software can do—such as support insurers on the analytical side as well—insurers are seeing that the system can help them with these problems, according to Norman.

Processing a claim is one of the most important transactional events between the insurer and the customer, points out Norman, and how well the insurer settles the claim is important.

“We’ve begun to have conversations with the claims folks as well as the catastrophe management folks to understand if there is a fraud issue or if you are sending adjusters in a timely manner,” he says. “If you don’t get your people out there quickly enough, customers tend to get more litigious and that ends up costing the insurer more money.”

A lot of insurers have mapping capabilities, but not the ability to combine maps with intelligence, points out Norman. He adds that some insurers have the intelligence and need the maps to see what’s going on.

“Being able to distribute that so companies can make decisions is drawing a lot of interest in the industry,” says Norman.

Global warming and climate change are subjects that insurers don’t like to talk about, adds Norman. He explains Pitney Bowes Software is not in the business of predicting climate change.

“That’s not what we do,” he says. “We enable insurance companies to better understand weather information that is available to them. There are a number of companies that provide accurate weather information and the insurer has to relate that to their policies and their exposures. That’s what we do. We are not in the climate business.”

Written by Default at 13:00

Is Intelligent Forecasting Part of Your 2013 Strategy?

Courtesy of Forbes

With debt troubles in Europe and economic woes in China holding back global economic growth, CEOs have to ask themselves, “Am I ready for what’s coming down the pike in 2013?”

One way to get ready is to deploy intelligent forecasting tools that help you strategize for the coming year.

What Is Intelligent Forecasting?

There’s no one hard-and-fast definition of intelligent forecasting, more widely known as business intelligence (BI). By and large, intelligent forecasting is a data analytics business strategy; it allows companies to obtain real-time intelligence via operational systems that warehouse data, analyze that data, and send “forecasting” results back through the pipeline to corporate decision makers.

From a CEO’s point of view, intelligent forecasting focuses on administration and analysis. Its tools bring together forecasting data gathered from operational administration sources, and merges them for further review and analysis by corporate managers, giving them better data to weigh before allocating resources to a given business initiative.

Make no mistake, business intelligence is big business, with data analysis the fastest growing sector in the BI market.

According to Gartner, the business intelligence market (including data warehouses and CRM analytics), has grown from a $57 billion market in 2010 to an estimated $81 billion by 2014. Intelligent forecasting, the data analysis side of business intelligence, accounted for $30 billion of the $57 billion in market assets in 2010. Gartner expects that number to grow significantly by the end of the decade.

“Analytics is the combustion engine of business, and it will be necessary for organizations that want to grow, innovate and optimize efficiency,” says Rita Salam, a BI analyst at Gartner. “Given its far-reaching impact, it is one of the few software markets that thrive even in adversity.”

Intelligent Forecasting in 2013

What trends in intelligent forecasting can CEOs expect for 2013? Here are a few good indicators:

BI and intelligent forecasting will overtake cloud computing. Cloud computing has had a nice ride, but data analysis forecasting will supplant it in 2013. That’s the conclusion drawn from a recent Gartner survey of 2,335 U.S. chief information officers. A separate survey from the consultancy outfit MorganFranklin says 75% of C-level managers say they will increase the BI proportion (or at least maintain current levels) of their data analysis budgets.

Geography matters. CEOs looking to leverage intelligence forecasting may wonder what geography has to do with data analysis. In a word, plenty. Industry insiders say there’s increased demand for analyzing data from a geographical viewpoint (think integrated mapping  for new business locations, or overlapping statistical data onto a digital map for more vertical analysis). The market has a term for that, and it’s called “location intelligence.” Expect to hear more about location intelligence in 2013.

BI goes mobile. Any corporate manager with an Android smartphone or anApple iPad recognizes the advantage of having critical analysis data delivered straight to one’s mobile device. With security anxieties over mobile devices abating, the delivery of key forecasting data to smartphones and tablets should be a rising trend in 2013.

Developing BI talent internally. C-level managers have historically relied on outside information-technology vendors to provide talent and expertise on business intelligence issues. But that could be going away. For starters, good data forecasting talent is rare – no surprise for a technology still in its infancy. Colleges are rapidly incorporating data analysis classwork into their technology curriculums, but progress is glacial. Increasingly, CEOs will be looking to develop intelligent forecasting talent in-house.

With intelligent data forecasting on the rise, corporate decision makers have a choice: adapt or be left behind. With data intelligence analysis already earning ROI credibility, corporate leaders had better get with the program – and fast.

Written by Default at 12:00

MAPCITE and Complexus Announce Partnership

Complexus to grow customer base for MAPCITE’s leading Enterprise GIS / Location Intelligence Solution in South Africa and Sub-Saharan Africa. 

MAPCITE is a location intelligence software company committed to helping organizations gain more insight from data, by repeatedly solving key problems for customers in the commercial, government and third sectors. MAPCITE and its products have earned a reputation for innovation, ease of use, speed, and the highest quality user experience.  As a result of their innovation with the Microsoft stack, MAPCITE are now members of the prestigious Microsoft Biz Spark One community.

With a growing list of clients on the African Continent and demand growing for location intelligence solutions, the partnership with Complexus is part of MAPCITEs’ strategy to expand into Africa and help client’s extract maximum value from geographical tagged data.   The MAPCITE product range takes the completely inexperienced "mapper" through simple to use and FREE applications, MAPCITE for Excel, through to the extremely powerful Web and On-Premises applications such as Microsoft SharePoint. For the more experienced Data and GIS user there are options for CRM BI Integration, Animated and Floor Plan Mapping, Specialist Mobile Phone Reporting Apps, Geo-Marketing and Massive Spatial Scalability plotting hundreds of millions of transactions in real time. 

Nick Bradshaw, Director of Complexus, said: “We are really pleased to be representing MAPCITE in South Africa and wider Sub-Saharan Africa.  It’s a perfect fit for our portfolio of tools that integrate with the Microsoft SharePoint platform.  We see many applications for the MAPCITE product and when integrated with the customers’ existing SharePoint deployment it can add value on many levels such as collaboration and sharing of location based data in the retail sector for example, right through to visualising exploration and survey data in the oil & gas sector.  Many clients in a diverse set of markets can benefit from using MAPCITE and we will be officially launching the product at SHARE 2013 conference, The Sandton Sun, Johannesburg, 11-13 March 2013” 

Adam Stevens at MAPCITE, “Our partnership with Complexus reflects the growing demand we see in Africa for GIS / Location Intelligence applications.  We already have a growing customer base in Africa and Complexus will help us service existing and new clients locally.  We are delighted to be working with Nick Bradshaw, benefiting from his considerable expertise in building markets for world-class business applications, as well as his knowledge of the African market.”

About MAPCITE

MAPCITE is a location intelligence software company headquartered in the UK.  MapCite's vision is to remove the knowledge and cost barriers that prevent most mapping projects from succeeding. MapCite's disruptive technology brings sophisticated Mapping Technology to everyone in an easy to use package of products that integrate with the Microsoft platform including Microsoft Excel and Microsoft SharePoint.   Already a member of the prestigious Microsoft Biz Spark One program, MAPCITE’S growing client base includes; British Red Cross, HP, Barloworld, GM, SAB Miller, Atlas Steel, AMEC, South African Airways, Thompson Reuters, Ford, PWC, ARUP, NASA, Deloitte, Petrobras & SA Health.

www.mapcite.com | info@mapcite.com | +44 (0) 845 6800004 

About Complexus

Complexus is niche software distributor / consultancy operating out of Cape Town that represents leading software brands in South Africa and wider Sub-Saharan Africa. The Complexus technology & service portfolio is aimed at organizations seeking to collaborate more effectively with enterprise content & project data, abstract maximum value from their investments in the Microsoft SharePoint platform and improve business processes.  Complexus offer clients software solutions & web parts that integrate with SharePoint allied to consulting services and training solutions based around ECM and SharePoint.  Complexus works with a mix of industry partners local to the South African market, is a BBBEE certified business & a member of the Microsoft Network ID 3766977.

www.complexus.co.za | mapcite@complexus.co.za | +27(0) 227720879

Written by Default at 16:00

bizspark1

Archive