Spatialytics is proud to announce the immediate release of Spatialytics ETL 3.0 and of GeoKettle 2.5, respectively the Enterprise and the Community Editions of its spatial ETL (Extract, Transform and Load) tool. Since July 2011 and the release of Spatialytics ETL 2.0 and of GeoKette 2.0, Spatialytics has performed a tremendeous work in order to provide users with an even more powerful, scalable, fast and aligned with the standards in the domain tool dedicated to operational and analytical geospatial data integration.
Here is the list of the main new features brought by this release:
In addition to all these new features and capabilities, various bug fixes and performance enhancements have also been performed. See the revision log for further details.
It has been a while since the last release of GeoKettle. The last version was indeed 3.2.0-20090609, mentioning that this release was done on June 09, 2009. Since this date, GeoKettle has migrated from a research prototype to a full fledged and professionaly supported spatial ETL tool. Year 2010 has indeed seen the creation of Spatialytics, the company which is now behind GeoKettle. Development activities that have led to version 2.0 have begun in October 2010 and have lasted till now, i.e. in July 2011. Version 2.0 of GeoKettle is thus the result of an important amount of work: many new and powerful features have been added, some bugs have been fixed, performance and robustness have been enhanced.
Even if the previous version of GeoKettle is numbered 3.2.0-20090609, the newest release is 2.0. This is because 3.2.0 was a reference to the version of Kettle on which GeoKettle was based. Current version is an important milestone for the project as it provides an important amount of new features, better performance and robustness. The previous numbering system did not allow to translate this matter of fact. That is why it has been decided to change the numbering of the versions and to name the new version as 2.0. It emphasizes more the important work performed to provide this new version.
However, it is important to note that versions 2.x will be the last versions of GeoKettle based on the Kettle 3.2 code base. Thanks to the tremendous work of the Kettle developers, future version of GeoKettle will be more pluggable with Kettle and will not be anymore a friendly and spatially enabled fork of Kettle. Hence, it will be possible to add spatial extensions provided by GeoKettle to any Kettle/PDI 4.x installation.
In addition to the classic zip archive containing the binary distribution of GeoKettle that you could download on the download page of the project, a multiplatform installer with a graphical user interface is now available. It is the easiest way to install GeoKettle on your computer.
While it was running perfectly on Linux 32/64 bits, the previous release of GeoKettle required to have a 32 bits JRE installed on your computer in order to work on Windows. With version 2.0, you can now run GeoKettle on this OS with a 64 bits JRE. GeoKettle has been extensively tested and is known to work very well on the Sun/Oracle JRE. There is no guarantee it will work on other JRE.
It was already possible to access the JTS objects contained in Geometry fields in the “Modified Java Script Value” step. This made possible the use of spatial analysis functions such as buffer calculations, overlays, metric operators, etc. With version 2.0, a dedicated step named “Spatial Analysis” has been contributed. It makes access to spatial analysis functions easier. Users can now perform buffer, intersection, union, centroïd, … computations through a GUI without typing any line of script.
It was already possible to preview rows processed during a transformation as a table in order to check if the transformation does the right job. With version 2.0 comes a geospatial preview. It is thus possible to access a cartographic view of geospatial data processed by a transformation. Select a step in the transformation, right click and select Preview in the popup menu. Select the geographic view tab.
You can display several layers (each layer corresponds to a geometry column in the data flow), modify colors, opacity and width of the symbols. You can also pan, zoom in/out and access attributes of a specific object.
One of the main new feature of GeoKettle is the support of OGR. GeoKettle is thus now able to read/write all geospatial (vector) file formats supported by OGR such as Mapinfo tab and MIF/MID, DGN, GML 2, GPX, DXF, KML 2, GeoConcept, Spatial/SQLite, GeoRSS, … For a complete list of formats, please visit the OGR supported formats page. Also note that as GeoKettle relies on OGR for reading/writing these data formats, same limitations apply.
When writing data with the OGR Output step, you can pass some OGR specific options. Please read OGR documentation for further details on each format.
OGR support is available not only in Spoon (the GUI-based job and transformation designer) but also in Kitchen and Pan, the two command line tools that come with GeoKettle and allow to run jobs and tranformations in batch mode (i.e. by calling directly the GeoKettle engine without the graphical interface).
In addition to the support of OGR data formats, GeoKettle 2.0 adds the read/write support of KML 2.2 file format. This enables the export and display of geospatial data stemming from GeoKettle into apps like Google Earth.
GeoKettle 2.0 also provides a read/write support for GML 3.1.1 file format. Together with the support of GML 2 provided by OGR, it enables for instance the processing of data stemming from OGC WFS request. This addition opens the door to the automation of numerous tasks in the management of a Spatial Data Infrastructure (SDI).
When reading/writing data in Shapefile, KML 2.2, GML 3.1.1 file formats, you are now able to manage the character encoding in which the data are read/written. It is particularly useful when you have to integrate data coming from different countries that use different encodings.
In Shapefile, GML 3.1.1, KML 2.2 input and output steps, it is now possible to specify the name of the file to read from or write to in a column of the input flow. You can also use variables to define this path. These capabilities open the door to the automatic processing of a batch of files in a single step of the transformation. It will be of a peculiar interest for the SDI administrators which need for instance to load an important number of files on a frequent basis.
With GeoKettle 2.0, you can now easily access to data stemming from sensors exposed via an OGC SOS (Sensor Observation Service) web service, as the open source implementation proposed by 52 North. You can then mix these data with different other heterogeneous (geo or not) data sources and enrich your analyses with live or historic data captured on the field.
You can access an OGC SOS via GET or POST method and specify different request parameters such as the time period bounds for which you want to retrieve measures captured by a specific sensor.
GeoKettle 2.0 allows to automate the harvesting or at the opposite the feeding of an OGC Catalog Service (CSW). Metadata collected from a CSW can thus be processed as any other data in a transformation. They can be used to pilot actions performed in a transformation.
The CSW Output step allows to automatically retrieve some metadata about the data produced/processed by the ETL tool and to disseminate them in a catalog service, such as the open source implementations provided by the GeoNetwork and MDWeb projects.
The dedicated steps in GeoKettle support different metadata formats/encodings such the one specified by ISO 19115/19139. It makes these steps of peculiar interest for persons involved in the setup and/or management of a Spatial Data Infrastructure (SDI), especially in the context of the European INSPIRE directive.
In previous version of GeoKettle, the SRS Transformation step was sometimes producing an inversion of coordinates axes when performing changes in the Spatial Reference System (SRS). This was due to a change performed by OGC in the definition of some SRS. This bug has been fixed in version 2.0 of your favorite spatial ETL tool.
SQL output generated through the SQL File Output step was buggy with PostGIS in the previous release. Geometry column were not properly created. It has been fixed in version 2.0 and you can now automate the production of SQL dumps that fully support the Geometry data type provided by PostGIS.
In a same way, with the SRID constraint enforcement in recent versions of PostGIS, insertion of geospatial data in this spatial DBMS was resulting in an error in previous versions of GeoKettle. SRID handling for PostGIS is now fully supported in GeoKettle 2.0. Insertion of rows and SQL output convey both the geospatial SRID.
In addition to the usual aggregation operators (sum, average, minimum, maximum, …) available in the Group by step, some spatial aggregation capabilities have been added. It is now possible to group data along with different field values and to produce a geometry which is the result of the union, the geometry collection or the enveloppe of all the grouped geometric features.
In version 2.0, the Calculator step has been extended with spatial computation capabilities. From now, you can perform usual spatial operations as computing:
Buffers Centroid Random point on surface Area Length Distance Intersection Union Envelope Boundary Convex hull Difference Symetric difference Inverse geometry
GeoKettle 2.0 supports JTS 1.12 and hence benefits from the last enhancements brought by this powerful library. It is for instance possible to compute single sided buffers with different caps and joins. Computation intensive tasks involving geometric features rely when it is possible on PreparedGeometry provided by JTS in order to drasticaly shorten the computation time.
Thanks to the mechanism for internationalization support provided by the Kettle developers, it is easy to provide the GeoKettle interfaces in different languages. By default, GeoKettle is fully available in English and French. Only parts of the tool are also available in other languages.
Obviously, we welcome any translations in different languages.
See the change log for further details about bugs that have been fixed and performance enhancements performed in this release.