A blog about my adventures with QGIS and other GIS in general.
Just a quick non-technical blog post.
My wife and I would like to introduce Harrison Mark Woodrow. The newest edition to our family.
Nothing has ever come close to how nervous I was yesterday morning, not knowing if Stace would finally be induced and we would have our little man. It was a long 13 hour labour but at 11:53pm (DST) 14/02/12 in Tweed Hospital, Australia little Harrison was born with all his cuteness.
All I can say is thank science for all the handy painkillers that have been created/made/found. That Epidural is one handy analgesia. Seeing my wife in so much pain before the Epidural was one of the hardest things I have ever had to the deal with. Ripping my heart out, hard.
Both Harrison and Mum are doing well.
Recently I have started reading two books, well one book and one collection of stories, on open source software and interaction design. This is all in an attempt to suck a little less at working on software, open source in particular i.e become a better person, programmer, and open source contributor.
The first book I started reading was About Face 3: The Essentials of Interaction Design. In my opinion this is a must read for anyone building software, or at least those who work with the UI and users.
One design principle the book covers is not leaking the Implementation Model into the UI. The Implementation Model is when the UI is designed around how the code works in the background, rather then the users mental model of the task. Most users don’t understand complex structures, or nested hierarchy, but yet we see it a lot within UI design due to it fitting the code design and the programmers view of the world perfectly well. Try explaining the branching, merging, and rebasing model of a Git tree to a non programmer and you will see what I mean.
That one design principle alone echoes strong with me, as lets be honest, most programmers are not UI designers and tend to do a pretty bad job at it, even me. When working on a feature the UI tends to be the last thing that is thought of and is just a quick interface for the code underneath.
Loss of orientation is another big thing. And what is the quickest way to get lost in a program? Dialog boxes! They popup, get in your face, most of the time have to be dismissed before you can see the results. Generally just a bad idea and people tend to get lost quick once you have more then one on the screen.
Those two principles alone are not going to make you a good UI designer but at least they give you something to ask yourself when working on a UI:
The second (free) book was brought to my attention by Brian on my Google+ feed, entitled Open Advice. Open Adivice is a collection of stories from people with differencing experience working on open source projects telling their stories on what they learnt and what they wish they had known when they started. The book aims to cover the answers to “”What would you have liked to know when you started contributing?”, which it does quite well.
It’s not heavy reading, but the story telling works well to bring home some of the things that everyone working on open source. Some of the stories cover things like; getting your first patch rejected; having a bad first IRC experience; writing good documentation; how to be a better community.
The book covers a range of topics so it’s a good read for everyone, regardless of your experience or knowledge area.
Those who don’t know history are destined to repeat it
So they are my two books for the start of 2012, hopefully they are a good read for you. I think in order to be good at anything you should strive, every year, to suck a little less at everything you do, even just a little bit. My role model in the software world is Scott Hanselman, generally a pretty cool dude, and has echoed a lot on his podcast this notion of learning sometime new every year to just get a little better at what you do, to become a better person and programmer.
So even if your not into learning or reading a lot, just reading these two books I can almost guarantee you will come out a better person for it.
Gary Sherman has just published a new Python plugin for QGIS that I think people will find very handy, I know I will. The plugin allows you to run Python scripts inside QGIS for tasks that don’t really require, or warrant, a whole plugin.
Go check out Gray’s post about the new plugin at http://spatialgalaxy.net/2012/01/29/script-runner-a-plugin-to-run-python-scripts-in-qgis/
The new plugin can be installed via the Plugin Installer using the “runner” or “script”. The Plugin Installer is another one of my favorite plugins for QGIS, being able to push out a new plugin and know that everyone can get it is a good feeling :)
The rule based rendering in QGIS has just got a make over to improve in some of the old usability issues it used to have. Most of the improvements are UI related. If you would like to try them out you will need to grab a copy of the latest dev build (qgis-dev in OSGeo4W)
Main improvements include:
The new rule dialog
As you can see in the screenshot, the rules are now organized in a tree which clearly expresses which rules should be applied and when.
In the example above, all the rules under the Sealed rule will only be applied if that rule is true. The old system would have you managing all rules in one big list and dealing with priorities in order to get the rules to apply right, the new dialog is a major improvement.
And the results! As you can see below, QGIS will only render the colored squares if the Sealed rule is true otherwise it just shows a green line.
The rules applied
The work was sponsored by Ville de Morges, Switzerland and developed by Martin Dobias. Thanks to both of them for these improvements.
More info:
Note: As this is a brand new feature there might be some bugs, or things that don’t quite work as expected. If you do find something don’t hesitate to file a bug report at hub.qgis.org so it can be fixed, or at least known about.
Support for MS SQL Server 2008 in QGIS is coming soon. A native QGIS provider for MS SQL Server is currently being worked on to make using, managing, and editing SQL Server data in QGIS just as easy as PostGIS.
The work is being sponsored by the Australian company Digital Mapping Solutions. So a very big thanks to them for this great feature!
There is no ETA on when it will be added to the main QGIS build, but the provider is currently in testing stage and hopefully will be in there soon.
So if you have been itching to try SQL Server data in QGIS, hang in there as a good solution is just around the corner.
P.S The other blog posts on this topic I used ogr, this method will still work fine after the native provider is added, however the native driver will add a nicer interface including integration into the QBrowser, better optimization for the QGIS code, and hopefully same feel as the PostGIS experience.
QGIS finally has expression based labels! (Although you must be running latest dev build)
What does that mean? Well QGIS used to be only able to label with a field from the layer, very limiting if you need to make a nice looking label string. Now you can use expressions (eg ‘Up ‘ || US_Invert || ‘some more text’ ) to label the feature, just like this.
Example of expression based labels
This is something that I missed a lot when moving from MapInfo to QGIS. After opening a ticket on the QGIS bug list and nothing happening with it for a couple of months (not that I expected anything to, everyone is busy enough as is. I don’t expect the devs to just jump at all my requests) so I decided I should at least attempt adding it myself. The joys of open source!
Turns out adding the expression labeling was the easy part, however there was no good generic expression string builder that I could use to build the expression string. QGIS already had the expression builder for the query window and the field calculator, however the code was very tied down to only work for those implementations, plus they didn’t scale with the increasing function list. I’m not going to go and make yet another dialog just for the labeling. Uniformity is the key to good user experience.
After searching around to see what other GIS systems did to get some inspiration it seemed that every example that I came across was, in my opinion, poor. Although there was one ArcGIS idea ticket that gave me a few ideas http://ideas.arcgis.com/ideaView?id=087300000008IbHAAU
So with that I started working on a generic expression builder that could be used to build an expression string anywhere, replacing the query window and field calculator in due time. One ring widget to rule them all, one ring widget to bind the, etc.
Generic expression builder
Key features of the new expression builder:
If the expression hits an error while you type you will be shown an “Expression is invalid” warning (yes I know it’s wrong in the screen shot). Clicking (more info) or hovering over the expression area will show you the error.
Expression has error
Searching can done by using the search box at the top. The function list will reduce to show only functions or fields containing that string (Note: it is case sensitive at the current time) .
Searching for a function or field name.
As with all programming it is never bug free so I expect, now that it is open to wider testing, that there might be a few that need to be addressed . There is also very limited function help written for the functions, although if anyone is willing to have a go at it I’m more than happy.
If you do find any bugs will the widget/dialog you can open a ticket at http://hub.qgis.org/projects/quantum-gis/issues , assign it to me and I’ll see what I can do.
I would like to thank Martin from the QGIS team for reviewing my code all though the project and helping me improve the code and idea. Also thanks to all the other people who gave ideas for the widget/dialog.
Enjoy!
I tell you. It always amazes me how much cool stuff you can do with great open source GIS software these days. GDAL is one of those great open source projects that I have just found a great use for (apart from just opening every raster type under the sun in QGIS).
GDAL has the ability to generate contours from a DEM, something that I have always wanted to try for my town but have never been able due to lack of a good DEMs.
Recently we purchased a set of DEMs that cover a large area as part of a study. Each DEM uses a grid size of 1mx1m. Prefect for generating contours.
First make sure that you have the latest version of the GdalTools plugin installed (GdalTools should be installed by default with QGIS. If it’s not, search “Gdal” in the plugin installer). Enable the plugin once it’s installed.
Load the DEM into QGIS using the Load Raster icon.
DEM loaded in QGIS
Now head up to the menu Raster->Extraction->Contour
Raster menu in QGIS
Select the settings that you need. For this DEM I am going to generate 250mm contours.
Contour dialog.
Take note of the text area at the bottom of the dialog as that is the exact command sent to GDAL in order to generate the contours. If you take a copy of that you can run it on the command line for batch processing later.
Hit ok.
250mm contours from the DEM
BAM! :)
Using QGIS for a one off DEM is fine and dandy but what if you have 3000 DEMs that you need to process. To hell with doing that by hand!
Remember the contour tool in QGIS told us the exact command line args to use, so creating a shell script for automating the process is pretty easy.
for f in *.asc do echo "Processing $f" gdal_contour -a ELEV -i 0.25 $f $f-250mm.shp done
The above code will loop though the current directory and process all the DEMs generating 250mm contours for each one. It saves each new contour file as {filename}-250mm.shp. You will need to change *.asc to whatever format your DEM is in.
Copy the above code into a file somewhere and call it generate_contours.sh. This can then be called from the command line using
sh generate_contours.sh
If you’re a windows user you will need to run OsGeo4W Shell in order to use sh.
Loading OsGeo4w shell.
Once the shell is loaded you can just call:
sh generate_contours.sh
Output from running generate_contours.sh
I ran the above sh script on a folder with about 2500 DEMs and it took around 4 hours to complete the whole folder. Of course performance will vary but it seems pretty quick considering.
Once again the possibly to use open source GIS tools to get my work done is amazing. No expensive software here.
So far I’m not aware of any line smoothing/generalizing abilities using GDAL/OGR. Although you can import the contours into GRASS and use that: http://grass.osgeo.org/wiki/V.generalize_tutorial
You can also generate the contours using GDAL via Python but that is a topic for another day.
Here is a quick video I did this morning using the very cool tool Gource. The video shows all the commits (1265) that were made between release 1.6 and 1.7. QGIS 1.7 was released on 19th June 2011 but there was still some clean up in the 1.7 branch paste that date into September.
It’s interesting to note the large burst of activity around November.
Last week I found a nice little undocumented feature of QGIS. I plan on writing documentation, so it won’t stay that way for long but I thought I would post about it first and run though it step by step.
This post is going to be a follow up post based on what Tim Sutton did for the same subject back in 2009 at http://linfiniti.com/2009/11/creating-dynamic-forms-a-new-feature-headed-for-qgis-1-4/
For data entry one feature I really like in QGIS is the automatic feature edit forms with support for textboxs, dropdowns and all sorts of other cool Qt controls to make data entry a breeze.
However one thing that people might not be aware of is that you can have a custom forms for data entry. QGIS will take care of setting all the fields and then saving the values back to your layer.
This could be handy if you want to have say a logo, some validation and maybe some text to help the user fill in the form correctly. Or just a custom form layout because you can.
One thing Tim didn’t follow up on was a post about how to add custom Python logic to the form, which I think is the coolest feature of having these custom forms.
So lets get started.
This process is pretty much the same as what Tim outlined in his blog post however I’m going to go over it again for completeness.
In order to create the custom form you will need to install Qt Desinger. For windows I haven’t found a way to just install the desinger although if you have QGIS installed it is normally installed with the Qt framework and can be found at C:\OSGeo4w\bin\designer.exe. If you’re on Linux you can run something like
sudo apt-get install qt4-designer
Ohh how I wish windows had a package management system :(
Fire up Qt Desinger and select “Dialog with Buttons Bottom”.
Lets throw on a couple of Labels and a few Line Edits for the data. Now set the form to use a Grid Layout (Right Click on empty space on form->Layout->Layout in Grid).
Now the trick in making a custom form for QGIS is naming the object the same as the field. So I my case I have a road layer with the following fields.
- Segment_ID
- Parcel_ID
- Name
- Alias_Name
- Locality
- Parcel_type
For my custom form I only care about Segment_ID and Name, so my form looks like:
Custom form in Qt Designer
Note that I have set the read only property of the Segment ID line edit to True so that it can’t be edited. I don’t want people messing around with the ID.
As I said above the tick is in the naming so right click on each line edit and select Change objectName, naming each line edit using the same name as the field. For me the first control is called Segment_ID and the other is called Name.
Make sure the objects are the same name as the field.
Save the form into a new folder, I have put mine in C:\Temp\Roads. Jump back into QGIS, load the properties dialog for the layer. Select the General tab and set Edit UI to the new form .ui file.
Setting the edit form UI file.
Save and exit the properties window. Enable the layer for editing (or not) and select an object with the Identify Feature tool.
Woot! Custom edit form in QGIS.
Magic! As I’m in edit mode any changes I make to the Name line edit will be reflected back on the layer (but not the Segment ID as it’s read only). If you are in non-edit mode then you are given the custom form with everything disabled and a cancel button.
Now creating a custom form like above is pretty cool although having some custom Python validation behind it would be even cooler.
What I want to do is add some validation to the Name field so the user can’t enter null road names.
First save your QGIS project (as the Python code runner will look where the project is saved for the Python module). Again I have saved mine in C:\Temp\Roads as Roads.qgs. Now lets make a new python file in your favourite text editor and add the following code.
from PyQt4.QtCore import *
from PyQt4.QtGui import *
nameField = None
myDialog = None
def formOpen(dialog,layerid,featureid):
global myDialog
myDialog = dialog
global nameField
nameField = dialog.findChild(QLineEdit,"Name")
buttonBox = dialog.findChild(QDialogButtonBox,"buttonBox")
# Disconnect the signal that QGIS has wired up for the dialog to the button box.
buttonBox.accepted.disconnect(myDialog.accept)
# Wire up our own signals.
buttonBox.accepted.connect(validate)
buttonBox.rejected.connect(myDialog.reject)
def validate():
# Make sure that the name field isn't empty.
if not nameField.text().length() > 0:
msgBox = QMessageBox()
msgBox.setText("Name field can not be null.")
msgBox.exec_()
else:
# Return the form as accpeted to QGIS.
myDialog.accept()
Wow! What the hell is all that! I’ll step though the code to explain each bit.
First import the modules from Qt and set up a few global variables to hold the dialog and name field.
from PyQt4.QtCore import * from PyQt4.QtGui import * nameField = None myDialog = None
Now we create a method that QGIS will call when it loads the form. This method takes an instance of our custom dialog, the Layer ID, and the Feature ID.
def formOpen(dialog,layerid,featureid):
Then using the findChild method we want to grab the reference to the Name field and the button box. We are also calling buttonBox.accepted.disconnect() to disconnect the slots that QGIS has auto wired up to our button box, we do this so we can hook up our own accepted logic.
After we have disconnected the accepted signal we can wire up our own call to the validate method using buttonBox.accepted.connect(validate).
global myDialog myDialog = dialog global nameField nameField = dialog.findChild(QLineEdit,"Name") buttonBox = dialog.findChild(QDialogButtonBox,"buttonBox") # Disconnect the signal that QGIS has wired up for the dialog to the button box. buttonBox.accepted.disconnect(myDialog.accept) # Wire up our own signals. buttonBox.accepted.connect(validate) buttonBox.rejected.connect(myDialog.reject)
We need a method to validate the logic. This will be called when the signal buttonBox.accepted() is called. The logic in this method should be pretty streight forward. If the Name line edit has a length > 0 then we accept the dialog, if not then we give the user a message and let them fix the mistake.
def validate():
# Make sure that the name field isn't empty.
if not nameField.text().length() > 0:
msgBox = QMessageBox()
msgBox.setText("Name field can not be null.")
msgBox.exec_()
else:
# Return the form as accpeted to QGIS.
myDialog.accept()
Now that you have a Python file with the custom validation logic we need to tell QGIS to use this logic for the form. First save the Python file in the same directory as your project. I have called mine C:\Temp\Roads\RoadForm.py.
Back on the General tab in the layer properties we can set the Init function field. We set this to call the module and function we just made. The syntax is {module name}.{function name}. In my case my module (the Python file we made before) is called RoadForm and the function is called formOpen, so it will be RoadForm.formOpen.
Set the Init function field to moduleName.functionName
Save and use the Identify Feature tool to select a feature. You shouldn’t get any errors if everything worked ok. Now delete everything in the Name field and hit Ok.
Validation in action
Sweet! The form can now not be accepted if the name field is null.
And that’s that. Pretty simple but powerful feature once you know how to set it up.
Enjoy!
If you do end up using this custom form with python logic stuff in the real world, leave a comment and maybe a picture. It would be good to see use cases for this cool QGIS feature.
Bonus
Why not add a red highlight to the textbox if something is not valid.
from PyQt4.QtCore import *
from PyQt4.QtGui import *
nameField = None
myDialog = None
def formOpen(dialog,layerid,featureid):
global myDialog
myDialog = dialog
global nameField
nameField = dialog.findChild(QLineEdit,"Name")
buttonBox = dialog.findChild(QDialogButtonBox,"buttonBox")
nameField.textChanged.connect(Name_onTextChanged)
# Disconnect the signal that QGIS has wired up for the dialog to the button box.
buttonBox.accepted.disconnect(myDialog.accept)
# Wire up our own signals.
buttonBox.accepted.connect(validate)
buttonBox.rejected.connect(myDialog.reject)
def validate():
# Make sure that the name field isn't empty.
if not nameField.text().length() > 0:
nameField.setStyleSheet("background-color: rgba(255, 107, 107, 150);")
msgBox = QMessageBox()
msgBox.setText("Name field can not be null.")
msgBox.exec_()
else:
# Return the form as accpeted to QGIS.
myDialog.accept()
def Name_onTextChanged(text):
if not nameField.text().length() > 0:
nameField.setStyleSheet("background-color: rgba(255, 107, 107, 150);")
else:
nameField.setStyleSheet("")
The key part of of this is nameField.textChanged.connect(Name_onTextChanged) and the Name_onTextChanged(text) method. Give it a try, I think it looks quite nice.
Change text background to red on invalid input.
Hopefully this tool can be of some use to people, as I know it has been very helpful to me since I made it.
As I’m a pretty heavy QGIS user now, and my work place still stores most, if not all, of our data MapInfo TAB format, one friction point for me using QGIS was having to restyle all the MapInfo layers. If we only had a handful of layer this wouldn’t be such a pain but we have a lot of tables and it would take me months to go though each one manually and style them.
I thought “there has to be some way I can automate this…” and so the MapInfo To QGIS Style Generator (or mapinfoToQgis.py) was born. Knowing that QGIS uses QML (a XML file format) to store it style information, and that MapInfo was able to export a style string for each object, I compared what QGIS generated for its QML using the same symbol I picked in QGIS as I had in MapInfo. Almost a 1 to 1 conversion! Once I worked out how to convert MapInfo point size to QGIS symbol size, and MapInfo colour value to RGB it was just a matter of generating a QML with the correct values.
Long story short, after a bit of clean up and writing a user guide I would like release version 0.1 of the MapInfo To QGIS Style Generator for wider testing.
Here is a quick example of the output.
Step 1: Take One MapInfo table.
Step 2: Run it though mapinfoToQgis.py
python mapinfoToQgis.py WaterFittings.Tab WatterFittings.qml -c FittingType --UseMapInfo
Step 3: Load QML file in QGIS
Result from running mapinfoToQGIS.py
Step 4: Get a beer?
If you are using MapInfo Font symbols or normal MapInfo 3.0 everything should come across almost exactly. mapinfoToQgis.py will use the same fonts in QGIS as you did in MapInfo and select to the correct symbol size. Although if you are using custom MapInfo 3.0 symbols you will get the default QGIS black square symbol,you can just change it to something better after loading the QML.
Currently the program only support converting symbols but I plan on adding line and region support sometime in the future.
The program can be found at https://github.com/NathanW2/MapInfo-to-QGIS-style-generator and more detailed instructions and download link can be found at https://github.com/NathanW2/MapInfo-to-QGIS-style-generator/wiki/Using-MapInfo-to-QGIS-style-generator.
Like I said at the start, hopefully other people will find this tool handy as I know I have. If you do find it handy let me know, I would love to hear peoples feedback. Also if you find any bugs let me know in the comments or log a issue on https://github.com/NathanW2/MapInfo-to-QGIS-style-generator/issues
Enjoy :)
|
Recent Comments