Up
Supercharged Online Data Access
History
BRERC data is mainly accessible via our own online apps. These leverage tailored middleware in a database (PostgreSQL) and application server (GeoServer). Some limited facilities existed for other users. But that is changing! In future we will offer comprehensive OGC standards-based options for all users. The GeoPackage format plays a key role in this.
Initiative
One or more customised GeoPackages will be created for each partner organisation. A zipped copy of this may be downloaded from our website for use locally in whatever way desired, for example in any modern GIS application (ArcGIS, MapInfo, etc). To unzip the archive the user needs an authentication key (authkey). Alternatively, Web Service requests can be made to any of the GeoPackage's layers, which are all published in GeoServer. In this case an 'authkey=...' parameter must be added to each request to authenticate. It isn't recommended to download large amounts of data by WFS but that is possible too. The only limitation is a scheduling restriction on the number of concurrent requests per user - to prevent server overloading. You can even develop your own apps incorporating standard WMS or WFS requests. Don't forget that GeoPackages are actually SQLite databases and can be optimised for your own application(s).
Technical
LIVE Services is designed for speed and security and therefore memory-based. For this initiative, which can involve large amounts of data, we add secondary storage on the frontend server. It is secured by kernel-based encryption. There is a folder for each user (BRERC partner). Each folder contains the GeoPackage(s), and their corresponding zipped version(s), for that user. Each morning the backend server mounts the secondary storage using SSHFS. It then executes a series of GDAL ogr2ogr commands to stream data into GeoPackages on the frontend server. GeoPackage layers can be generated from PostgreSQL tables and/or views, or from other GIS file formats (Shapefile, MapInfo TAB/DAT, etc). The backend server then uses SSH to remotely zip up the GeoPackages and creates links to both in the appropriate places on the frontend server. These are the GeoServer data directory and the website download folder. Finally it publishes each GeoPackage in GeoServer, that is, it creates a 'featuretype' for each layer in the GeoPackage, using the GeoServer REST interface. This is possible because a GeoPackage can be treated as a 'store' in GeoServer. Once the backend (database) server has set up the GeoPackages, off-peak, it is then freed up to concentrate on its primary role - supporting applications. Just one more thing ... the website and GeoServer will also need updating whenever the frontend starts up, because these settings are not persistent.
Web Services
These include WMS (Web Map Service) and WFS (Web Feature Service). They are quite different, but complementary. WMS delivers raster tiles which make up a map image already styled on the server. Typically you can then click on the map to get detailed feature info. WFS is feature-orientated at the outset, delivering geometry and associated attributes for local processing, including complex geospatial analysis. Here these services return exactly the same data that you can download in a zipped GeoPackage.
First check out the GeoServer WMS reference and WMS vendor parameters, and the WFS reference and WFS vendor parameters. Read up on 'cql_filter' if you have anything more than a passing interest in Web Services.
The GeoServer 'Catalog Mode' is set to HIDE to ensure that users only see featuretypes (layers) that they have access to. This is crucial because there will be many users and therefore many GeoPackages, each having many layers, and not all layer names will be unique.
My WFS App
The purpose of this app is two-fold. To demonstrate the value of the GeoPackage/GeoServer combination. And to provide a practical starting point for use of this facility.
Enter your authkey to view the layers available to you. There is now a 'demo' authkey for prospective users. Just accept the default at the prompt.
Click to start
The 'Web Services' summary page contains a URL which you can copy-and-paste into clients such as ArcGIS, MapInfo or QGIS to make WMS requests. Then it lists the featuretypes (layers) you can browse via WFS right here in this app. For each one selected the next page shows a request URL and a response. This is the heart of the app. Read on for more details.
The request/response pair is accompanied by a number of links. Those that modify the request URL also re-submit the request so the response is refreshed automatically. Examples of this are the sort and find links, which provide a convenient way of customising the feature set returned. To clear any ordering and filtering parameters added to the request just restart the app using the link provided, or simply refresh the page.
Paging requires a starting offset (startIndex) which is 0 initially, and a page size (count). Here we have one feature per page. Use the back and next links to browse the feature set. They work in a circular fashion - the first feature goes backward to the last, and the last goes forward to the first.
The response consists of attributes and a geometry. Because the geometry may be very detailed there is a show/hide facility. In show mode there are two options for spatial filtering - limiting the feature set by geometry using the intersection function. You can specify the actual geometry to use for the comparison, or you can specify another feature (typically in a different layer) whose geometry you want to use. For the first option click the spatial link, which will allow you to paste in the geometry. First copy a geometry to the clipboard, from this app or elsewhere. For the second option (usually because the actual geometry is too complex) click the cross-layer spatial query link. You are prompted to enter layer (which must be in the same GeoPackage), attribute to use for matching, and filter value. For example 'sssi; name; Ashton Court'. The query is performed by a GeoServer extension which takes the first match found so be careful with the filter value.
You can copy-and-paste the request into another app or a browser. Edit the request URL directly this way.
You can download the current feature set, although this is not recommended.
You can issue web service requests in many computing languages. Check out the source code of this app to see how it is done in JavaScript.
Example 1: What is the largest SSSI by area?
Go to the Web Services page of My WFS. Click OK at the authkey prompt to use the 'demo' GeoPackage.
Click GetFeature for demo:sssi.
Click sort for attribute ha (hectares) to go to the first (smallest) site.
Click back to go to the last (largest) site, which is Severn Estuary.
For further info check out the Natural England links included in this feature type.
Example 2: Download all Small Tortoiseshell records for Blaise Castle Estate.
Go to the Web Services page of My WFS. Click OK at the authkey prompt to use the 'demo' GeoPackage.
Click GetFeature for demo:small-tortoiseshell.
Click find for attribute Place and enter 'Blaise' for the text to find.
Click sort for attribute Year_Date to get a rough chronological ordering.
Click download to create a 'small-tortoiseshell.csv' local file.
Up