“Your wife might leave you if you work with open source” – No more

pros_cons_programmerThis is what an ESRI distributor told me (more or less) referring to the amount of time it takes someone to set up an interactive map web site  built around open source software.

Was meant in jest of course. He did have a point though. For anyone that is not very technical savvy, trying to create a simple website with a map and some interactivity was… let’s say, challenging the least. Compare having to install various components, editing Apache configs, copy/paste files around, searching for plugins, writing code for simple map interactions like zoom in/out, with the clean, drag & drop way of Web AppBuilder for ArcGIS.

To me, this was always the big gap in the FOSS4G ecosystem. Desktop GIS like QGIS, were not a problem any more. Download and install via a wizard. Easy. But there was never an quick (or painless) way of creating a map website.

Not until now, that is. Boundless has announced the release of their QGIS 2.14.1 distribution (1) which includes, among other goodies, the Web App Builder (currently in Beta) plugin.

Web App Builder allows you to create web apps directly from QGIS. Without writing a single line of code.

You can add different base maps or overlays:


And select the tools/functionality you want available on your site:


There are lots of other customization settings but those were the most interesting ones in my opinion. There are a few bugs (the tabbed interface is not working, doesn’t work well with different SRIDs) which result in blank pages, but all in all, I had a fully functional web site in less than 5 mins. That by itself is a huge improvement from the past!

You can download the Boundless QGIS distribution here. There is a form where you have to enter your details, etc but its definitely worthwhile.

Lots of kudos to the guys in Boundless and  I am sure lots of families will be thanking them too 😉

(1) To avoid confusion or misunderstandings as per @Mr Fahrenheit’s comment: QGIS is an open source project created and still actively been developed by an international team of contributors and not by just Boundless. This post is referring to a specific distribution/branch of QGIS which includes the WAB plugin


Creating custom forms in QGIS – Part 2

Better late than never they say…

Anyway, on the previous post I explained how to customize the QGIS edit form using the available widgets depending on your attribute data types. In this post I will continue on how to further customize your form using the Drag and Drop Designer in QGIS (form layouts may look slightly different from the previous post as I now use QGIS 2.2 version).

Open the Layer properties form, click on the Field properties and select [Drag and Drop Designer] from the [Attribute editor Layout] drop down:


To create your custom dialog, you first need to create a ‘category’ i.e. a tab. So  click on the green ‘+’ button and add the label that will appear on your tab.


Now you can start adding fields inside this tab using the right blue arrow. Make sure you highligted the attributes you want to add AS WELL AS the tab name in the right panel for the arrow to get enabled.


You can add more tabs and fields repeating the steps above, as shown in the following screenshot:


And the completed form should now look like this:

The Streets attributes and Photo tabs…


…and the Website tab


Pretty simple- right?


Creating custom forms in QGIS – Part 1

One of the things I love about QGIS is its ability to easily create custom forms. Usually without writing a single line of code. The QGIS manual in my opinion doesn’t do it justice, as it only describes this functionality very briefly, so I decided to write a couple of posts explaining the process with examples.

I will be using a shapefile containing a road network. The attributes in the shapefile are as below and there are pretty self-explanatory:


Now, note the [Edit Widget] field. Its default value for each attribute is “Line Edit” which basically means it will display a simple text box for the user to enter a value when in Edit Mode. Obviously, this is not ideal as we got values like number of Lanes (LANES attribute) which should be numeric values or dates (DATE_OPEN attribute). QGIS won’t validate the form values till after we press the save button.

So first thing we need to do is change the widget (control) that will display in the Edit form for the DATE_OPEN attribute. Press the [Line Edit] button for this attribute and the attribute edit dialog will open. Select [Calendar] from the left panel:


You can edit the date format to control how the date will be displayed. When finished click the OK button.

Next, I wanted the ROAD_CLASS attribute to be selected from a drop-down list. To do this, follow the previous steps for the ROAD_CLASS attribute and this time select [Unique Values] as the widget. The Unique values widget will display all unique values that already exist in the shapefile in a drop-down list. And as we also need to be able to enter new values, check the [Editable] check box (the second one- the first is checked by default and it controls whether the field would be editable or not in edit mode).


Similary the Road environment (ROAD_ENVIR attribute) should be displayed as a drop-down list but this time we will use our own values. To do this select the [Value Map] widget and enter your values in the displayed grid as shown below:


Then, for the LANES attribute we would use the [Edit Range] widget. Number of lanes can range theoretically between 1 and 6, so put these values as min, max and define a step of 1:


But we can also display images or web pages in our form. Not surprisingly the photo widget is called [Photo] and you can defined the width and height of the image to display…


…whereas to display a web page of specific width and height you select the [Webview] widget:



You can also display a color ramp in your form. To do this, use the [SYMBOL] field in this example and select the [Color] widget:


If you followed this far, congratulations! Now close the properties window and start editing. Click on the identify button and click on any of the features in your shapefile. In the identify window right-click on one of the features and select the [Edit Feature Form] option:



Have a look at the form and note how the different fields display different widgets/controls based on wh

at we set them at, in the previous steps


As some of you may have noticed by now, this is not all the widgets that are available in QGIS but hopefully we covered quite a few of them and you got the general idea.

But this is not all. With QGIS, you can further customize your forms, adding tabs and grouping your controls. That’ll be next, in Part 2





How to make your first QGIS custom form on Windows

For a long time I wanted to give a go at creating a QGIS plugin. Nothing in particular, just a ‘Hello World’ plugin or similar so I could understand how the whole QGIS/python thing hangs together. Seasoned readers of my blog may have noticed by now that I mainly (only?) work on Windows OS. And let me tell you, trying to find exactly what is needed in terms of python libraries for being able to create a simple form in QGIS proved to be a big challenge. .

But I have figured it out and those are my notes in case you also want to give it a try. They don’t go as far as creating a plugin. They merely create a custom form you can use for editing a layer. But I consider this a HUGE step for me nevertheless!

I was running QGIS 1.8 on a Windows 7 64-bit laptop. I also had ArcGIS 10 installed and for Python development I use PyScripter. This posed an interesting problem of how to use PyScripter for both ArcGIS and QGIS but I had solved this earlier

So next, I apparently had to use PyQt4 for being able to create forms within QGIS. But DO NOT use the version for Python 2.7 from this link as -as I found out at my expense- its not compatible with QGIS. The safest way is to use the OSGeo4W Installer. Select the Advanced install and select to download and install the packages as shown in the next screenshot. Not entirely sure that ALL these are required but they work for me so….

py_packagesNote the last item on the list (SIP) which the version I have is the previous from the current one. I *think* you need this version otherwise you get runtime errors when trying to load the PyQt4 libraries i.e. “RuntimeError: the sip module implements API vXX to vXX but the PyQt4.QtCore module requires API vXX

Yes- it was a very trial and error job…

Moving on, you can then open PyScripter for Python 2.7 and check if you can load the PyQt4 libraries by running on the python prompt:

>>> from PyQt4.QtCore import *
>>> from PyQt4.QtGui import *

If all goes well you should not get any errors. To be on the safe side and ensure you have the same setup in the QGIS python environment (it may sound an unecessary step but trust me, its not) run the above command on the QGIS python window as well.

And so, finally, you are ready to make your custom form. But for this I simply followed Nathan’s example from his QGIS blog

Next step would be to create my Hello World plugin. On a next post….


Configure PyScripter to use with QGIS (and still use arcpy) on Windows

As many of you may already know, PyScripter is an excellent IDE for Python. For those of you who haven’t tried it… well, you don’t know what you are missing! I was using PyScripter quite happily with ArcGIS 10 and arcpy without ever having to tweak anything in PyScripter. All was well until I decided to explore PyQGIS, QGIS’ scripting language. I could not, for the life of me, get this initial statement to work in PyScripter although it was working fine in the QGIS Python console:

import qgis.core

I was getting the “usual” error:

Traceback (most recent call last):
File “<interactive input>”, line 1, in <module>
ImportError: No module named qgis.core

Now, arcpy is using Python 2.6 whereas QGIS (I was using the latest 1.8 Lisboa version) is using 2.7. One would have thought that since PyScripter allows you to work with different versions of Python this wouldn’t be such a big deal. Yet it was.  Below I am outlining the steps I took to solve it.

One thing to note here is that I installed QGIS 1.8 using the standalone windows installer. Things may have been different if I used the OSeo4W installer which apparently sets the environment for you….but I can’t be sure.

One of the problems with the standalone installer is that it installs Python 2.7 but does not create any registry entries- which is what PyScripter mainly uses. So the first thing to do is to create the relevant registry entries for 2.7. Assuming you have installed QGIS in the default folder (C:\Program Files (x86)\Quantum GIS Lisboa for a 64bit machine), you can use this registry settings file (edit the file accordingly if using 32-bit Windows).

Next, copy the python27.dll from C:\Program Files (x86)\Quantum GIS Lisboa\bin to c:\Windows\SysWOW64 (or \system32 for 32-bit machines)

Finally, create this batch file to start pyscripter with (note you may change the PROGRA~2 to PROGRA~1 for 32-bit machines.

set GDAL_DRIVER_PATH=%OSGEO4W_ROOT%\bin\gdalplugins\1.9
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.2\lib
path %PATH%;”%OSGEO4W_ROOT%\apps\Python27\Scripts\”
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGISPATH=%OSGEO4W_ROOT%\apps\qgis
start “PyScripter” /B “C:\Program Files (x86)\PyScripter\PyScripter.exe” –python27

Happy coding!

Setting codepages in shapefiles to display in ArcMap 10 – A saga

I spent a good few hours last week trying to get ArcMap 10 to display a shapefile with Greek data in. Not an easy job. The shapefile was Open Street Map data including all place names in Greece, downloaded from Cloudmade. When I opened the attribute table in ArcMap it looked something like this:


On the other hand, the shapefile was displaying fine in QGIS IF you opened it by setting the Encoding to UTF-8. So I set forth trying to find a way of specifying the encoding in ArcMap. To skip right to the bottom, I can tell you now that nothing I tried using ESRI-only tools and suggestions worked. But during my quest I found some useful bits of information which may be prove helpful for someone in the future: So this is my story:

I first made sure that my Regional Settings was set to Greek/Greece and the same was true for the Language for Non-Unicode programs- just in case. (this used to work for Arcview 3.x – obviously ArcMap IS a Unicode program):


Then I tried adding the *.cpg file. This is basically a single-line text file with the same name as the shapefile which stores codepage information for the attribute data (if the shapefile does not carry this information). Tried various codepage codes. It didn’t work.

I then move on to add some registry values according to this little gem: HowTo:  Read and write shapefile and dBASE files encoded in various code pages. Nope.

Getting desparate, I then tried the 30th byte way from ESRI which tells you how to determine whether a shapefile has a code page or not (apparently this information is stored in the 30th byte)::

To find the 30th byte, you count the sets of characters in the center. In this example, you start with 03, which counts as 1. Count over 30, counting only the character sets shown in blue in this example. If the set is 00, the code page is not set. The 30th character in this example is 0E. Therefore, the code page is set.

0B8D:0100 03 64 02 07 01 00 00 00-A1 00 41 00 00 00 00 00 .d……..A…..
1489:0110 00 00 00 00 00 00 00 00-00 00 00 00 00 0E 00 00 …………….

If the code page is not set in the .dbf header, you can create a code page file (.cpg) to store the code page. To create a code page file, you use a text editor, such as vi or Notepad, add the code page identifier for the shapefile to the file, and save it with a .cpg extension in the same location as the other files that make up the shapefile. You have to be sure you know the encoding used for the shapefile so you place the correct code page in the .cpg file.

If for some reason you have a .cpg file and the code page is set in the .dbf, the information in the .dbf header takes priority when importing a shapefile. If no code page is set in the .dbf, the code page is read from the .cpg file. If the code page is not set in the .dbf and no .cpg file is present, the code page of the current locale of the operating system from which shp2sde is being run (the server where ArcSDE is installed) will be used.

So I opened the dbf in a HexEditor. It did have a value on the 30th byte. I put it to 00. Re-opened shapefile in ArcMap. Same rubbish characters.

By that time I had enough. I opened the file in QGIS (with a UTF-8 encoding) and then saved it under a new name with a CP-1253 encoding. And of course it worked. Which was what I thought to try in the beginning but my masochist self wanted to find a solution without using any 3rd party tools.

I mean, admire the simplicity of it all:

Step 1. Add the shapefile and set encoding


Step 2. Save it under a new name


Now, how difficult it would be for ESRI to have an encoding option when opening a shapefile?

If you think I missed something and there is indeed a much simpler way (using ESRI tools) to view attribute data in ArcMap I would be more than happy to learn about it!