INTRODUCTION

Five years ago this article could not have been written. At the time of writing, (April, 2014) sufficient free resources have become available to allow the independent researcher to make use of a Geographic Information System (GIS) as good as any commercial system, and which avoids paying a cost of many thousands of pounds in license and maintenance fees.

In SIAH Volume 43, Part 1, George Barlow published his GIS based article on "The Interrelationship between Freemen and Manors in Late Anglo-Saxon Suffolk". He explained how "the use of a GIS platform to accurately plot the information permitted greater quantities of data to be analysed across larger regional areas at a greater magnitude of focus than had previously been possible."

I hope to demonstrate that such a GIS platform is now freely available, at no financial cost, to the independent researcher who is willing to undergo the fairly steep learning curve involved in the successful use of such a computerised mapping system.

COMMERCIAL GIS SYSTEMS

ARCGIS

Esri's ArcGIS suite of products is the dominant GIS product on a global scale, and has 36% of the world market. Version 8 was released in 1999, and was a reworking of a number of older products which had begun development in the USA in 1969. A personal computer (PC) version was released in 1986. Commercial and Institutional Licenses for the full product with maintenance costs well in excess of most individuals ability to pay, although ArcGIS for home users is now available for a charge of 100 USD a year. Many UK Universities use ArcGIS, and lecturers, undergraduates, MA, and PhD candidates obtain free use of this programme through their university email account and password systems.

ArcGIS uses a vector file format known as a shapefile, which is probably the nearest thing to an industry standard for GIS files. It is recognised by the computer file extension .SHP or .shp.

MAPINFO

MapInfo was founded in the USA in 1986 to develop a lower cost GIS tool for PC users. In 2007 the company was acquired by Pitney Bowes. As with Esri, both companies have continually improved and updated their products. The MapInfo suite of programs is widely used by UK local authorities, including within Suffolk and Norfolk. However, owing to the costs involved with installing newer versions, each local authority is potentially implementing a different version of the suite. There is no home user version available at present. MapInfo file types which you may encounter as a QGIS user are .TAB, .MIF and .MID.

OTHERS

There are other commercial GIS systems available, but ArcGis and MapInfo are the ones which an East Anglian researcher is most likely to come across.

WHAT CAN A GIS PROGRAM (note 1) ACTUALLY DO?

The most compelling feature of these mapping programs is map geo-location. A series of sheets or "tiles" of calibrated digital maps can be loaded into the program which will be able to line them up correctly adjoining each other. In addition, by moving the computer's cursor over any feature, the user may read off the map reference of the point in question. In order for this to work correctly the files being used must include the appropriate geo-location information. Digital maps supplied by the Ordnance Survey to its institutional users, all include this geo-location information in one form or another.

By using these maps as a backdrop, the user can add new layers of information, such as the lines of sewer pipes for drainage engineers, or the location of archaeological sites and finds for an Archaeology Service user. These new layers of information can be saved together with their own geo-location information.

Layers from a whole range of users and sources can be combined within the GIS program to show relationships which might otherwise go unnoticed.

Although that is an extremely valuable function, the GIS can also carry out a range of analysis of the data presented to it, giving results such as all the schools within an area prone to flooding, or even within a set distance from such an area. The mapping and analysis of historic data is also capable of being carried out as shown by George Barlow. Actually producing the final maps is usually fairly easy, once the necessary learning curve has been scaled. Obtaining the raw data to be included within any map is usually the most laborious of the processes involved.

The data attached to a map is contained within what is known as an Attribute Table. In computer terms this is usually saved within a .DBF file type.

A FREE AND OPEN SOURCE GIS PROGRAM

QGIS (note 2)

A program called QGIS, formerly known as Quantum GIS, is now the predominant freely available GIS platform worldwide. QGIS is a cross-platform free and open source desktop geographic information systems application that provides data viewing, editing, and analysis capabilities.

QGIS development began as an initiative by Gary Sherman in 2002, but he was later joined by other developers eager to work on developing an open source GIS. It was not until 2009 was it was developed sufficiently to justify its release as Version 1.0. By September 2013, a considerable number of improvements had resulted in the release of Version 2.0, with much improved cartographic capabilities, and a re- worked interface. It is intended to release three new versions in each year, with Version 2.2 already released in February, 2014, and codenamed "Valmiera".

QGIS 2.2 can be expected to perform all the functions of the commercially available programs, and it is now in use by a number of public and private institutions across the world, including the UK and Europe. It is attracting considerable local authority interest as an adjunct to commercial products, and in some cases, as a replacement. The latest version of QGIS can be easily downloaded from the project website. (note 3)

SOURCES OF FREE MAPS

Having decided to make use of QGIS, the new user typically wants to load some maps to get a feel for the program. QGIS itself can be downloaded along with some example maps and an introduction to their use. These example maps are based in Alaska, as much USA sourced mapping data has been freely available for many years.

In Britain, the public had to wait until 2010 for good quality free digital maps to become available. On 1 April 2010, the Ordnance Survey launched OS OpenData, the service that made a selection of mapping datasets for Great Britain available for commercial or non- commercial use without restriction.

Users discovered with some disappointment that these were not copies of the well loved 1:50,000 or 1:25,000 maps which were already in widespread use on paper sheets. OS OpenData was made up of a series of completely new maps, designed to be distributed freely, but excluding many of the details already contained in previously published maps. At first these were somewhat experimental, but by 2014, a series of 11 products has become available free by filling in an order form on the OS website. (note 4)

Each vector product is available in the form of shapefiles for ArcGIS, or as MapInfo files, at the user’s choice. Raster files come as Geo-TIFFS, or with world files for geo-location. These files can all be downloaded, but, if prospective users only have a slow internet connection line, then the products can be delivered to your door by post, on a number of DVDs, totally free of charge. This is a truly remarkable level of service.

CHOICE OF MAP

The Ordnance Survey offers a number of different maps, and it is quite daunting to decide which product is required. Users may have them all, but when it comes to using them, there is a choice to be made based upon the prospective end product desired by the user.

If one is going to work at a regional or County wide level, then the product known as Strategi® is useful for a regional overview of road networks, railway lines, cities and rural wooded areas. Rivers and the coastline are also included, although OS does not mention these on their Order Page. Strategi works well at around 1:250,000 scales.

Strategi is a vector map product, while the equivalent raster product is called "1:250 000 Scale Colour Raster" and is equivalent to the former OS Travel Map, but in an image format.

For work on an individual town, OS Street View® is marketed as a generalised and simplified street level map, ideal for city-centre plans. However, it is available solely as a raster file at scale 1:10,000.

On the other hand, OS VectorMap™ District, despite its name, can be supplied as either a set of vectors, or as a set of rasters. The nominal viewing scale is 1:25,000, with a recommended scale range of 1:15,000 to 1:30,000.

The OS provides User Guides in .PDF format for its various products, but these can only be found by searching the OS website.

DO I WANT RASTER OR VECTOR PRODUCTS?

Digital maps exist in two completely different formats; rasters and vectors. A raster map is a digital image made up of pixels just like a digital photograph. If users attempt to "zoom in" too far to see more detail, there will come a point where the image breaks up into a blur. Raster maps have the advantage of being in a fixed OS produced style, so they look good straight away. However there is very little extra detail available to a GIS program in most rasters, other than the map image which can be seen.

Raster maps are useful for providing backdrops over which the user can trace off desired features, or can be overlaid by another map or vector layer.

Vector maps are at the heart of a GIS system. A vector is a line which can be blown up to any scale and it retains its integrity. It functions in the same way as the output from a drawing or illustration program such as Adobe Illustrator CS®. A vector map is typically built of many layers of image information. Typically there may be a coastline layer, a rivers layer, perhaps several road layers, and a layer of settlements for example.

In QGIS a vector layer will be one of three types; either a layer of points such as towns, or of lines such as rivers, or as polygons such as a boundary enclosing a field or estate.

For each layer there will be an Attribute Table, which for rivers would probably contain the name of each river and possibly a classification into main river, secondary river, minor river etc. This table can be accessed within QGIS, and amended if necessary. Data analysis can also take place on the attribute table. Each layer can also be styled by the user.

When a new layer is loaded into QGIS the program allocates a random colour to lines, or a coloured dot to any points. It is up to the user to amend the style as he or she thinks fit. Do not despair if upon first viewing of a newly loaded vector file it looks a complete mess. This is normal. When the user zooms in or out to the correct scale the mess will not look so bad, and one can begin to change the colour schemes and fonts as appropriate.

WHAT FILE FORMAT DO I NEED?

It is usually at this stage of the process that the new user runs into another of the knowledge issues which proliferate around GIS. One may be a QGIS user, but the Ordnance Survey website only mentions file formats for ArcGIS and MapInfo users. I recommend choosing the Esri ArcGIS options, as shapefiles (.SHP remember) are not only the most widely known GIS vector file format, but are the native file format for QGIS. (QGIS can also read MapInfo files, but the process can become complex.)

For raster products you can safely download the Geotiff format, which is readable in QGIS, and QGIS will place the maps into the correct geographic orientation.

Some OS OpenData raster products are available as TIFF-LZW format. (LZW simply describes the compression method used in creating the .TIFF file, and is not included in the file extension.) QGIS users can safely use these TIFF files as well, although there is a need to use world files with them, (format .TFW in this case) to achieve correct geo-location within the QGIS program. Although these files are supplied by the OS within the same download, they are packaged in separate folders. It is up to the user to copy these files into the same folder as the .TIFF files in order to achieve geo-location within QGIS.

These free datasets from the Ordnance Survey are extremely welcome, but it must be said that they are far from easy to comprehend, or to use. But eventually beginners should be able to extract some useful mapping information from them.

When using these OS products you should include the following acknowledgement:
‘Contains Ordnance Survey data © Crown copyright and database right 20YY’.

ONE LAYER BUT SEVERAL FILES

When you acquire a digital map layer from whatever source you should make sure that you receive any other files associated with it.

The simplest are the raster files as they are largely self contained. There is a special type of .TIFF file that is called a Geo-Tiff. This is a .tiff file which has been modified to include the necessary geographic information for the GIS program to properly geo-locate it. This is the only raster format which can include this information. You need to be aware that if you process such a file with Photoshop, or other image processing program, these programs usually remove the geo-location data as they do not recognise its presence.

Other .TIFF files may be supplied with separate so-called world files which contain the geo- locating data. You recognise this second file as it contains the same file name as the raster, but the file extension will be .TFW, or a variant of this.

World files are an Esri ArcGis format, but are recognised by all GIS programs.

Rasters supplied by the Ordnance Survey can be either geo-tiffs or tiffs with corresponding world files.

Rasters supplied by a MapInfo user will have an accompanying .TAB file, rather than an Esri worldfile.

Vector files are more complex because of all the information needed to make them work. A MapInfo file might be a .TAB file but is also likely to be in the MapInfo interchange format. This is a .MIF file for the vectors and .MID file containing the geo-location details and others. Once again you will find that the filenames must be all the same, and only the file extension changes.

This convention allows the GIS program to load all of the associated files, without the user having to specify them all individually.

A shapefile, with the suffix .SHP, will be accompanied by, at least, a geolocating file, (.PRJ) a positional index of the feature geometry file (.SHX) and an attribute file, (.DBF), which contains all of the data associated with the map layer.

MORE ABOUT STYLES

The use of a style file containing information which allows QGIS to draw rivers etc in the correct thickness, and with the correct colour, has already been mentioned.

Until March, 2014, QGIS users of OS OpenData had to invent their own style files to view OS vector files. Otherwise those files appeared as a mass of lines and dots, in colours allocated randomly by QGIS.

Happily, the Ordnance Survey wish to support the QGIS community and have invested considerable resources in producing QGIS specific style files. QGIS style files have the file extension .QML, although QGIS can read other types of style file provided they contain the necessary data.

Looking for these .QML files on the OS GitHub (note 5) website given below is still a laborious process, so some patience is required.

OTHER SOURCES OF FREE MAPS

EDINA

University users who have privileged free access to ArcGis, also have free access to the many datasets supplied to higher educational users by EDINA. Although funded at a national level, EDINA operates through the University of Edinburgh.

Although these products are not available to the independent researcher, EDINA also operates a website called ShareGeo Open, which contains resources available to everyone free of charge.

Many of these datasets are of only specialist interest, but also included are re-workings of some of the OS Opendata material to make them more user friendly. It is well worth browsing through the 130 datasets currently available at ShareGeo Open. (note 6)

THE BRITISH GEOLOGICAL SURVEY

The BGS provides free access to UK Geological data at a scale of 1:625,000, which is suitable for work at a regional or county level. Both Bedrock and Superficial geology layers can be downloaded. (note 7) Once users are a little more experienced, they can also access BGS Web Map Services using the QGIS Layer/WMS facility, where geology layers are available at a scale of 1:50,000.

OPENSTREETMAP (note 8)

Believe it or not, there is a free and Open Source alternative to Ordnance Survey maps called OpenStreetMap. This has been built by volunteers world-wide using hand held GPS devices, and is also accessible through QGIS. It uses its own unique file structure, and results at present require a bit more than beginner knowledge to use successfully within QGIS.

UK DATA SERVICE (FORMERLY ESDS)

You will need to register with this service, but if you need a particular dataset, this is a small price to pay.

One very useful mapset for the local historian available here is the old ESDS map of nineteenth century parish boundaries. (note 9) Find it listed as follows:
SN 4828 "GIS of the Ancient Parishes of England and Wales, 1500-1850", which is a development of this work:
"SN 4348 Historic Parishes of England and Wales : an Electronic Map of Boundaries before 1850 with a Gazetteer and Metadata - Kain, R.J.P., University of Exeter. School of Geography and Archaeology".

The GIS version download covers the whole country, but users can select only the counties required from within QGIS. Data was collected from the old one inch (1:63360) maps so is good for scales around 1:50,000 scale.

THE QUESTION OF SCALE

If a raster file is viewed at the wrong scale it becomes quite obvious when details available within the map have become lost. Vector maps do not degrade in quite the same way.

In theory a vector map can be viewed at any scale, but in some ways this presents a trap for the unwary. A vector map is usable only within a decent range of the scale at which it was created. For example, a coastline drawn from a 1:1,000,000 base map will not contain enough suitable detail to view it at 1:25,000. Conversely a map drawn at a town scale of 1:2,500, will contain a mass of jumbled lines when viewed at even 1:50,000 scale.

MAP PROJECTIONS

If you have been used to using maps on paper sheets, you may never have needed to consider the map projection being used. In GIS programs a lot of work has been put into coping with maps presented in different projections. In GIS parlance a projection is often referred to as a Coordinate Reference System or CRS, because a CRS refers to a projection together with the datum from which the map coordinates are measured.

If the world is envisaged as a globe with an image of the surface drawn on it, it can be seen that transferring the image to a flat surface results in a number of distortions. Cartographers have attempted to address this problem by re-projecting the image in a number of ways, but no method can produce correct results at every location.

Therefore different projection methods have been adopted across the world to achieve a 'best fit' for local circumstances. In fact, the world is not a perfect globe, but is elliptical from pole to pole, and also elliptical around the equator. This has caused there to be a large number of local map projections in use around the world.

The oil and gas companies, with large sums available to back up their exploration programmes, were the first to attempt to codify this wide and diverse collection of map projections. Modern GIS programs have adopted the petroleum industry standard list of reference systems, known as EPSG codes, and include these within their software. (EPSG is derived from the initials of the now defunct European Petroleum Survey Group.) Also included in all GIS software are the necessary algorithms to convert maps from one CRS to any other.

Both the traditional projection name and the EPSG code are used within QGIS.

MAP PROJECTIONS QGIS USERS NEED TO KNOW ABOUT

WGS 84 - EPSG 4326 WORLD GEODETIC SYSTEM

The Global Positioning System (GPS) uses satellites to provide navigation data which is used routinely by drivers who own in -vehicle systems. The map projection used is known as the World Geodetic System (WGS) which was designed to provide accurate results anywhere in the world. The specification was updated in 1984, and the current CRS is therefore called WGS 84. The EPSG code for this projection is 4326.

WGS 84 is also the basis for the views provided by Google Earth and associated maps. WGS 84 uses latitude and longitude coordinates, but the prime meridian is about 100 metres east of the Greenwich Meridian.

In order to use Google Earth to import images or maps into QGIS, users need to be aware that if they are to be used in conjunction with UK OS maps, then they need to transformed, or re-projected, into the OS National Grid CRS.

OSGB 36 - EPSG 27700 10 THE BRITISH NATIONAL GRID

The British National Grid has a long and complex history. The current system did not come into use until after 1945, but was based upon a new national triangulation of the whole of Great Britain begun in 1936. Hence the short code name OSGB 36. The national projection is based upon a modified form of the Transverse Mercator projection, known today as the Airey Ellipsoid.

The well known grid square system known as the National Grid also dates from this period. This grid does not correspond to the lines of latitude and longitude of the Transverse Mercator graticule, but is superimposed over it. The National Grid is based upon metres measured from a datum point located south west of the Scilly Isles.

We have become used to using the OS system of employing two letters to describe the 100 kilometre square in which each mapsheet resides. Thus Suffolk is contained within squares TL (previously 52) and TM (previously 62), and a further six digits are used to make a map reference accurate to a one hundred metre square.

At present, QGIS cannot recognise our traditional two letter grid references, but uses the absolute number of metres from datum, making use of the conversions from, for example, TL to 52 as above. Thus TL86, the ten kilometre square around Bury St Edmunds, becomes located at cordinates 580000,260000 in QGIS.

MAPS FROM THE ORDNANCE SURVEY MADE PRIOR TO 1945

Ordnance Survey maps in the nineteenth century were based upon a Cassini projection, but there were a variety of measured triangulations in use in different parts of the country. 10 In practice that meant that 1:2500 map sheets for a county like Suffolk were kept consistent within the County, but were unlikely to produce an accurate alignment with a neighbouring county like Cambridgeshire. These maps were drawn up to the county boundary, but not beyond, so any boundary change also threw the system into disarray.

The old County system of mapping persisted into the mid twentieth century, and GIS users of maps dating from before the Second World War will usually find some issues around accurately matching the features shown on these maps with the position of features shown on later maps.

Given a big enough scanner, old maps of any type can be scanned in, using a normal raster format such as .TIF or .JPG. Such a file can then be geo-referenced within QGIS usually by plotting prominent landmarks or known coordinates against a modern map. Drawbacks often encountered relate to the probably unknown nature of the Projection and datum used in the old map, and results can also be affected by many years of distortion to the paper sheet by annual variations in heat and humidity. It works, but results can be variable.

There is a specific problem arising from such old County Series maps which have been calibrated using MapInfo. Although they are correctly located within MapInfo, when such a .TIF file with its associated .TAB file is read into QGIS, it is unlikely to be placed correctly. A sample of twelve 1:2500 tiles from the 1884 County Series from around Bury St Edmunds has been tested in QGIS. Of the twelve tiles, only three would properly geo-locate. This problem has also been noticed by staff at the Ordnance Survey, but at present this issue remains unresolved.

OPEN SOURCE TECHNICAL TERMS

Although QGIS is available for computers using Microsoft Windows, Apple OS, Linux and its variants, as well as experimentally on Android tablets, its developers frequently use open source jargon and acronyms within the documentation, and within answers to queries.

QGIS itself is built to make use of most of the Open Source Geographic products already in the public domain. You will hear references to GRASS, GDAL, OGR, Sextante, Python, etc., all of which are built into QGIS.

As you use QGIS no doubt you will come to know something about these names. You can look up their origins by the use of a Google enquiry.

The QGIS program itself can be extended by the use of 'plug-ins' developed by individuals for their own use, but made available to everybody. A pool of plug-ins is maintained, and these are easily loaded into QGIS when required. (note 11) The most popular plug-ins will probably end up being absorbed into the main QGIS code base. Unfortunately these plug-ins tend to suffer from a lack of a proper description of the function which they perform, but this issue is being addressed.

AVAILABLE SUPPORT FOR THE QGIS USER

On the QGIS website 3 there is a section entitled 'Documentation', where the user can find the QGIS Manual, a QGIS tutorial and a Gentle Introduction to GIS. These are the primary resources for learning about this powerful program.

Users will also become aware of the QGIS UK users Group, (note 12) a large number of YouTube videos, and a number of Blogs written by individuals, which are drawn together in a website called Planet QGIS. (note 13) Because QGIS has developed so quickly it is as well to check that the video or blog is referring to a recent version of the program.

Even the Ordnance Survey has a YouTube channel, (note 14) which includes introductions to OS Opendata, and Ordnance Survey also run day courses in major cities to demonstrate the use of OS OpenData. These courses always contain a major concentration upon the use of QGIS, which the OS is keen to promote.

When users run into a problem, there are a number of online fora where these issues can be discussed. QGIS has its own forum (note 15) for users, but there are a number of other Open Source GIS fora to be explored.

Local authorities and institutions which may be investigating the use of QGIS will find that there are several UK based consultancies which specialise in GIS applications using both the commercial GIS programs and QGIS. These consultancies often write special purpose 'plug- ins' for clients, which then become available to the whole QGIS community.

MAP EXAMPLES