Something that I need to do now and then is generate points along a line at supplied distance. I had never really looked into doing it in QGIS until this question poped up on gis.stackexchange.com. This is a quick blog post because I thought it was a pretty handy little thing to do.
In the development version there is a new method on QgsGeometry called interpolate. This method takes a distance and returns a point along a line at that distance. Perfect! We can then just wrap this in a loop and generate a point increasing the distance as we move along
from qgis.core import (QgsFeature, QgsGeometry,
QgsVectorLayer, QgsMapLayerRegistry,
QgsField)
from PyQt4.QtCore import QVariant
from qgis.utils import iface
def createPointsAt(distance, geom):
length = geom.length()
currentdistance = distance
feats = []
while currentdistance < length:
# Get a point along the line at the current distance
point = geom.interpolate(currentdistance)
# Create a new QgsFeature and assign it the new geometry
fet = QgsFeature()
fet.setAttributeMap( { 0 : currentdistance } )
fet.setGeometry(point)
feats.append(fet)
# Increase the distance
currentdistance = currentdistance + distance
return feats
def pointsAlongLine(distance):
# Create a new memory layer and add a distance attribute
vl = QgsVectorLayer("Point", "distance nodes", "memory")
pr = vl.dataProvider()
pr.addAttributes( [ QgsField("distance", QVariant.Int) ] )
layer = iface.mapCanvas().currentLayer()
# Loop though all the selected features
for feature in layer.selectedFeatures():
geom = feature.geometry()
features = createPointsAt(distance, geom)
pr.addFeatures(features)
vl.updateExtents()
QgsMapLayerRegistry.instance().addMapLayer(vl)
The above code might look a bit scary at first if you have never done any Python/pyqgis but hopefully the comments will ease the pain a little. The main bit is the createPointsAt function.
Cool! If we want to use this we can just stick it in a file in the .qgis/python folder (lets call it pointtools.py) and then run import pointtools in the python console.
So lets have a go. First select some objects then run the following in the Python Console
import pointtools
pointtools.pointsAlongLine(40)
That will generate a point every 40 meters along then selected lines
Distance nodes along line in qgis-dev
To generate nodes along different lines, select the new features, then call pointtools.pointsAlongLine(40) in the Python console.
Simple as that!
(Who knows, someone (maybe me) might even add this as a core object function in QGIS in the future)
Atlas is a super handy little QGIS plugin that lets you generate a map series from a composer and a coverage (grid) layer.
The authors of the plugin are now looking for some funding to make it a core part of QGIS. Making it part of QGIS would mean integration into the composer and maybe APIs for developers to add extensions.
The authors are looking at raising 7000€ in order top get the work done, but the more money raised the more features that are added.
Version note: This will only work in the latest dev build of QGIS – not in 1.8
The lack of uni for the next couple of weeks has left me some time at night to work on some features that I really wish QGIS had. One of these features was better date and time support in the expression engine. Date and time is an important concept when working on inspection data and not being able to style my features in QGIS using date operations was bugging me. So in good open source fashion I added some.
Here are the current functions (more to come in the future):
$now – returns the current date and time
age({datetime},{datetime}) – returns the difference between the two dates
todate({string}) – converts a string to date type
totime({string}) – converts a string to time type
tointerval({string}) – converts a string to a interval type (details below)
day({datetime} or {interval}) – returns the day from a datetime type or the number of days in a interval.
hour(…) – Same as above but for hours
minute(…) – Same as above but for minutes
second(…) – Same as above but for seconds
day(..) – Same as above but for days
week(..) – Same as above but for weeks
month(…) – Same as above but for months
year(…) – Same as above but for years
{datetime} – {interval} = {new datetime} – returns a new datetime subtracting the interval
{datetime} + {interval} = {new datetime} – returns a new datetime adding the interval
The interval type
Functions like age(..),tointerval(), {datetime} -/+ {interval}, day(..), hour(..), etc, use, or return, Intervals. An Interval is a measure of time that we can use for different things. An example of an Interval is ‘1 Year 2 Months’ this is then converted to a number of seconds and used for any calculations.
For example one can take away 10 days from the current date by doing the following ( -> marks the output ):
todate($now - '10 Days')
-> 2012-06-20
as
todate($now)
-> 2012-06-30
We can also do something like:
todate($now + '2 Years 1 Month 10 Days')
-> 2014-08-10
The age() function will return an interval which we can use extract what information we need.
The number of days between two dates:
day(age('2012-06-30', '2012-06-10'))
-> 20
-- Think of it as '2012-06-30' - '2012-06-10'
-- Note: day(), month(), etc, functions return doubles so you can get
-- 21.135234 days if you are using date & time type rather than just date type
-- wrap the result in toint() to get a more sane output.
Day() will also work on a plain date:
day('2012-06-30')
-> 30
We can even get the number of seconds between two dates:
Currently the only date format supported is {year}-{month}-{day} as seen in the examples above. Shouldn’t be too hard to add support to the todate(), todatetime(), totime() functions for giving it a pattern to use when converting the string e.g. dd-mm-YYYY, or something like that.
More on this fancy new stuff
When I wrote the new expression builder dialog a while ago I made it dynamic so that any new functions added to the expression engine will show up automatically. So here they are:
List of new date and time functions.
We can also use these functions in the rule based rending, which is where the power really comes in. Lets see something like that in action:
Styled using days and years
Should be pretty straight forward to understand. We are using the age() and day() functions to style the events that are older than 30 days, within 30 days, for today, or in the future. We also check the year of the event using year() and year($now) to make sure we only see this years events, or style them differently depending on if they are last years events or in the future.
This is the result of the above rules:
Result of using date functions in rule based renderer
I’m also using the date functions in the expression based labelling to label the features using the following expression:
CASE
WHEN year( "dateadded") < year($now) THEN
'Last Year'
WHEN day(age("dateadded", $now)) < 0 THEN
day(age("dateadded", todate($now))) || ' Days old'
ELSE
day(age("dateadded", todate($now))) || ' Days to go'
END
Well that’s it. Hope you find it handy in your day-to-day mapping. I know I will be using it a lot.
Thanks to Martin and Jürgen for the code reviews during the process; venturing in an unknown part of the code base always makes me nervous but that is all part of learning, and sometimes you can make some pretty cool stuff. Some other random notes: The general idea has been modelled of how Postgres handles dates and times, it’s not an exact copy but follows the same kind of ideas. The interval class also uses the same number of seconds for one year that postgres does so that we can be consistent with the output.
So this years first uni semester is done and dusted, now I have some free time. Blog all the things!
This is a follow up post for discussion that was started on LinkedIn about showing features older, or newer, then a certain date different colours The main post was about using free, or low cost, solutions in order to aid in mapping water networks. I recommend that everyone watch it. A very good presentation.
I recommended that you could use the rule based rendering engine but the expression engine in QGIS doesn’t have any date functions yet. All good we can add them if we need and once I get my head around the expression engine I plan on doing exactly that. But for now we can do it a different way.
We are going to use Spatialite, but any database will do (syntax and process will vary).
Lets have a look an inspection layer we have in our Spatialite database viewed in QGIS:
Pretty boring and hard to see what has been done in the last 30 days. Now with the lack of support for dates in the expression engine we have to use another methods. For this example we will use the really handy DBManger plugin that now ships with QGIS from 1.8. Load it, connect to your database, and run the following query:
SELECT id,
CASE WHEN DATE("date_checked") > DATE('now', '-30 days') THEN
'Within 30 Days'
ELSE
'older'
END as age, date_checked, geom
FROM "inspections"
As you can see anything that is within the 30 days now has the “Within 30 Days” string in the age column, or else it has “older”. CASE statements can be very powerful things in SQL sometimes.
Now load it into QGIS, style, and label it using the new age column
You now have a layer that is style based on age but is also dynamic. Adding a new inspection point will now will be styled according to those rules. (Although you will have to edit the normal inspection layer with it turned off as views/queries are not editable – without some setup anyway)
It might be a simple thing to some but sometimes it’s hard to find the right words to describe what you want when you are looking for this kind of thing. So hopefully this has helped a few people get started with visualizing their time/date based data in QGIS.
Here is the change log of all the new stuff in 1.8:
– QGIS Browser – a stand alone app and a new panel in QGIS. The browser lets you easily navigate your file system and connection based (PostGIS, WFS etc.) datasets, preview them and drag and drop items into the canvas. – DB Manager – the DB manager is now officially part of QGIS core. You can drag layers from the QGIS Browser into DB Manager and it will import your layer into your spatial database. Drag and drop tables between spatial databases and they will get imported. You can use the DB Manager to execute SQL queries against your spatial database and then view the spatial output for queries by adding the results to QGIS as a query layer. – Action Tool –now there is a tool on the map tools toolbar that will allow you to click on a vector feature and execute an action. – MSSQL Spatial Support –you can now connect to your Microsoft SQL Server spatial databases using QGIS. – Customization –allows setting up simplified QGIS interface by hiding various components of main window and widgets in dialogs. – New symbol layer types –Line Pattern Fill, Point Pattern fill – Composers –have multiple lines on legend items using a specified character – Expression based labelling – Heatmap tool –a new core plugin has been added for generating raster heatmaps from point data. You may need to activate this plugin using the plugin manager. – GPS Tracking –The GPS live tracking user interface was overhauled and many fixes and improvements were added to it. – Menu Re-organisation –The menus were re-organised a little – we now have separate menus for Vector and Raster and many plugins were updated to place their menus in the new Vector and Raster top level menus. – Offset Curves –a new digitising tool for creating offset curves was added. – Terrain Analysis Plugin –a new core plugin was added for doing terrain analysis – and it can make really good looking coloured relief maps. – Ellipse renderer –symbollayer to render ellipse shapes (and also rectangles, triangles, crosses by specifying width and height). Moreover, the symbol layer allows to set all parameters (width, height, colors, rotation, outline with) from data fields, in mm or map units – New scale selector with predefined scales – Option to add layers to selected or active group – Pan To Selected tool – New tools in Vector menu –densify geoemtries, Build spatial index – Export/add geometry column tool can export info using layer CRS, project CRS or ellipsoidal measurements – Model/view based tree for rules in rule-based renderer – Updated CRS selector dialog – Improvements in Spatial Bookmarks – Plugin metadata in metadata.txt – New plugin repository – Refactored postgres data provider:support for arbitrary key (including non-numeric and multi column), support for requesting a certain geometry type and/or srid in QgsDataSourceURI added gdal_fillnodata to GDALTools plugin – Support for PostGIS TopoGeometry datatype – Python bindings for vector field symbollayer and general updates to the python bindings. – New message log window – Benchmark program – Row cache for attribute table – Legend independent drawing order – UUID generation widget for attribute table – Added support of editable views in SpatiaLite databases – Expression based widget in field calculator – Creation of event layers in analysis lib using linear referencing – Group selected layers option added to the TOC context menu – load/save layer style (new symbology) from/to SLD document – WFS support in QGIS Server – Option to skip WKT geometry when copying from attribute table – upport for zipped and gzipped layers – Test suite now passes all tests on major platforms and nightly tests – Copy and paste styles between layers – Set tile size for WMS layers – Support for nesting projects within other projects
Thanks to all the sponsors and everyone who put a lot of work into this release!
Something that I always find interesting is how people are using different open source tools to get their work done. This post attempts to outline how I/we are using QGIS at work for different projects.
Kerb mapping, condition, and defect pickup
This project is currently being done by a 67 year old foreman who has worked for the council for a very long time and has great knowledge of the town. QGIS, with the main working layers stored in PostGIS, was setup so that he can:
Digitize kerb lines from aerial photos.
Split the existing kerb lines into segments depending on different asset rules.
Give each segment an overall condition rating.
Add defect points along the each kerb segment e.g. broken, lifted, etc,
Each defect point is snapped to the underlying kerb line and chainages (distance along line) is generated using a update statement at the end of the project (could be done using a insert trigger if needed) using ST_line_locate_point(line, point).
Defect points coloured by risk captured against the kerb line
Overall QGIS has been great for this project. The built in data entry forms have been a great help to allow fast and correct data entry. Each form has four drop downs all with present values and descriptions to aid in data entry.
Flood damaged claim maps
We recently suffered, like the rest of Queensland, some really major flooding which caused large amounts of damage to our road infrastructure. We got off pretty light compared to some places, nevertheless we still had a lot of damaged assets. And so began the process of collecting data that could be used for state government funding claims.
Anyway, onto the QGIS bit. QGIS was installed on one of the main engineers computers in order for him to make maps for each claim. Having the ability for him to have one map window but multiple frames in the composer helped him to create multiple views of the same data with ease.
In total there are 42 QGIS project files with a main project file which served the base layers to the other projects, using the cool Embed Layers and Groups feature. This means any change in main base project was reflected up(down?) to the other projects next time they are opened. The main project file has things like, property layer; normal road layers, with labels; road layer with roads for claims. The other 42 projects have a filtered, and styled, road layer to only show roads in that batch, and its composers (print layouts).
Normally we would use MapInfo for this kind of thing but consider this: There are at least 3 print layouts per claim, each layout could have more then one map frame. Now with MapInfo only being able to have a 1:1 ratio between the map window and the map frame in the layout you would need at least 3 map windows per claim. Quick calc:
42 * 3 = 126+n map windows + 126 print layouts (n = extra map frames in layouts)
Each map window has its own copy of every layer, making change once apply every where changes hard. This of course doesn’t apply to styles as they are stored in the .map (tab) file, but does for labels, style overrides, etc. I’ll pass.
QGIS is no means perfect for printing or print layouts but the 1:N map window to map frame ratio worked really well for this project. The styling options in QGIS also helped to change the display of the map depending on what was needed to be shown quickly, one even used the rule-based rendering.
You get the point.
Moving on.
Processing GPS photos with road chainages
This one I am quite proud of. It’s nothing fancy but still saves a lot of time. While not really QGIS only but a combination of QGIS+Spatialite it process GPS photos and assigns them a road name and chainage.
The issue: A large influx of GPS photos for the different flood damage projects and the need to process them quickly so that they got assigned to the correct road and chainage. Now you can map GPS photos easy enough but then you still have to go to each one and assign a road name, chainage, and move it into the correct folder. To hell with doing that by hand, this is why we invented GIS.
The result is a little (140 line) python script that:
extracts the coordinates from each photos,
finds the closest (within tolerance) road distance node (distance nodes are generated at 5m intervals along the road, around 800,000 in total for the whole shire),
gets the road name, locality, and chainage for that node,
creates a folder with that road name,
renames the photo with {name} @ {chainage},
moves it into the road name folder it is assign to.
inserts a record for that photo into the spatialite database that can be viewed in QGIS.
The Spatialite database has a spatial index on the road distance nodes and with that in place it can process 148 photos in 8 seconds. Not too bad at all. Now all we have to do to process the photos is stick them into a special folder and run process.bat.
Porting our planning scheme maps
I have been involved in creating, and maintaining, our planning scheme maps for the council. It’s been a pretty fun project, apart from the constant moving target that is the state planning specifications, but I digress.
Planning scheme in QGIS
This project was done, and still is, in MapInfo. While there is nothing technically wrong with that, it has become a bit more of a pain to maintain then one would hope. The planning scheme is not just one map but rather a series of different maps all with different scales and requirements. I’m sure by now you can start to see the issues that can arise:
No dynamic scale bar for layouts (not even a scale bar object rather just text and boxes made to look like a scale bar. With no group items feature moving these around is a pain).
1:1 map window to map frame means excessive map windows when the data is all the same with just different views.
Legends don’t support ordering, adding items, or groups.
With no embedding base maps feature like in QGIS it’s hard to change one thing and apply it to all the map windows/workspaces.
The specifications also ask for lines with symbols along them to show things like bikeways, footpaths etc, something that can’t be done in MapInfo, well it can by using the line style editor but I would rather stab myself in the eye.
The one thing I haven’t fully worked out how to do in QGIS yet is fully automate the printing process. Currently I open MapInfo using a batch file and pass it a workspace and MBX which prints the layouts and exits. I do this for each map type. In QGIS I have a few options:
Create a plugin that runs though each project and prints off its composers.
Create a python script that runs from a batch file using qgis.core and qgis.gui QGIS python bindings.
add a –code option to the command line of QGIS so that you could run: qgis.exe –noplugins –code “print.py”, which would open QGIS and run the python code and exit.
I’m yet to explore what option is the best for this project but I’ll get back to you on that. Once I have the above issue sorted I plan on creating the maps in QGIS to see how it would turn out (time permitting)
Custom asset data collection program
This one would have to be my favourite. I really love programming (most days), and being able to create our own data collection program using QGIS and MS SQL 2008 has been great.
While it is only very very young I’m already seeing some great potential. Using an open source base (apart from MS SQL) has given us a lot of power, power to change stuff that we don’t like (which so far has been one minor bug), and the power to get exactly what we need.
I can’t talk about this project a lot as it is only very new and still only in design/testing/prototyping stage.
The main things for me are:
Ease of use. If I get asked how to do something over and over I have failed the users. And no 100 page training manuals.
Fast
No menus, or right-click menus! I’m a power user and even I hate navigating menus on a tablet.
Easy to build custom forms
Online/Offline syncing
Ease of use. Oh did I say this already!? Well it’s important.
Easy to configure by admins.
Limited use of dialogs. It’s NOT ok for an app to ask users to confirm 100 dialogs to do one thing.
Overall I think using QGIS and PyQt I can hit all the targets listed above quite well. In fact I know I can because I have already hit most of the them in the last couple of weeks.
Summary
So that is my list of QGIS uses in my local government situation, hopefully it wasn’t TL;DR and you found it interesting. I’m sure there will be plenty more to add at the end of 2012.
One thing I didn’t explain very well in my other post was how to correctly set up value binding between your custom form and QGIS. I didn’t explain it because at the time I didn’t know how.
The other day I was building a custom form QGIS for a project I am working on. I had named all the fields right, set the ui as the edit form for the layer, but only the line edits were getting bound to the correct values.
So having a dig around in the code I noticed that QGIS uses the same methods to bind the built-in edit forms as it does for the custom forms, meaning that you must set what kind of control you want to use in Layer Properties -> Fields
Correctly binding values
First create the form with the controls you need, remember to name them the same as your fields.
Custom form with controls using the same name as the fields
Note that here I have a QComboBox with the FeatureCla name, this will bind the combo box to the FeatureCla field in my dataset in QGIS.
Now set the custom form as the Edit UI for the layer
Set the Edit UI to your form
Tip: You can use relative paths if you store the form along side your project file
Flick to the Fields tab and set up the Edit Widget type for each field that you have used on the custom feature form.
Set the Edit Widget that matches your control
I have set the FeatueCla field to use Unique values widget, this tells QGIS to collect all the unique values from that column and add them to the QComboBox. There are a range of different edit widgets you can set
Each will map to a different set of control types (Widgets) e.g. If you want to have a checkbox on your form you must select Checkbox in the Edit Widget list to get it to bind correctly.
Save the properties and head back to you map. Use the Identify Tool to select a feature.
Values bound to form
And that is it. Pretty cool hey!
Final thoughts
This is one feature I really like in QGIS. The ability to create custom forms for people to do data entry without the need to build a plugin is very cool. Couple this the built-in GPS module for QGIS and you have yourself a nice simple field data collection program.
I have some ideas to make this feature even more powerful, but more on that later once I get some time to add it in.
Ok the title is a bit of a lie. QGIS did support MS SQL Server 2008 before by using OGR but this is a native provider so it’s a lot more integrated..
Good news everyone!
QGIS now has a native MS SQL 2008 provider. The provider can found using the new toolbar button (purple icon) or in the MS SQL node in the QBrowser tree. The provider also supports drag and drop import.
This addition will open QGIS up to a whole new set of users who have to use MS SQL but love QGIS.
Currently this is only in master but I will be in the 1.8 release when it comes out.
Note: At the moment you have to have a geometry_columns table in the database in order to connect, this table is the same format used by PostGIS and can be made by importing a layer using the ogr2ogr method. There will be a fix coming for this at some stage.
Come join the discussions on your preferred platform. If you prefer not to say much, social networks are still a great way to keep up to date with all the cool stuff happening in QGIS land.
If you have something cool that you have done with QGIS and would like to pimp it to the world, free feel to contact me, or even just mention QGIS in the post, Mentioning QGIS will make it show up in the search on Twitter or Google+. I keep a keen eye on all sites for anything cool that I can reshare on the main QGIS pages.
Gary Sherman has just published a new Python plugin for QGIS that I think people will find very handy, I know I will. The plugin allows you to run Python scripts inside QGIS for tasks that don’t really require, or warrant, a whole plugin.
The new plugin can be installed via the Plugin Installer using the “runner” or “script”. The Plugin Installer is another one of my favorite plugins for QGIS, being able to push out a new plugin and know that everyone can get it is a good feeling :)
Nathan Woodrow 