The SIG Initiative
The Head Office of Geodesy and Cartography is currently realizing the following projects:
- GEOPORTAL 2,
- "Georeferential Database of Topographic Objects (Georeferencyjna Baza Danych Obiektów Topograficznych – GBDOT) with a national management system",
- TERYT2 – national register of borders and areas of territorial division units,
- "Information System for Protecting the Country against extraordinary threats" (Informatyczny System Osłony Kraju przed nadzwyczajnymi zagrożeniami – ISOK),
- Integrated Real Estate Information System,
- Geodetic Territorial Development Network.
The above mentioned project will result in development of IT solutions to support the activities of the Head Office of Geodesy and Cartography.
Information systems of the Head Office of Geodesy and Cartography (GUGiK)
Having noticed the need to manage multiple projects realized on various levels, and responding to the identified project needs, we have noticed a chance to obtain optimal integration of the solutions developed.
An initiative called SIG (Information Systems of the Head Office of Geodesy and Cartography - Systemy Informacyjne GUGiK) has been launched, with an aim to elaborate and implement a common framework for all the projects realized by the Head Office of Geodesy and Cartography. This initiative has been divided into 3 main areas:
Main areas of the SIG initiative
In the area of providing a coherent architecture of SIG, the following activities have been launched:
business layer integration, which led to establishing of:
- a homogeneous catalogue of business services,
- a map of services,
- a map of products,
- homogeneous documentation,
- architecture principles,
- Architecture Council and Architecture Office,
IT layer integration, which led to establishing of:
- a static model of SIG,
- integration scenarios,
- unified requirements models,
- a services sizing method.
development of the existing infrastructure, ensuring optimal usage of the existing infrastructure.
The architecture of the solutions has been prepared in such a way, that in case new guidelines of digitalization are introduced, the Head Office of Geodesy and Cartography will be able the adjust its IT solutions to the new guidelines with no delay.
A homogeneous realization method for initiatives launched under SIG includes:
- compliance to the PRINCE2 methodology,
- usage of a set of guidelines in the areas of: quality, configuration, risk and communication management,
- tools to support organization, architecture and maintenance.
A homogeneous maintenance method for SIG will be achieved by means of:
- creating a coherent maintenance model, based on ITIL best practices,
- providing business services with defined parameters,
- ensuring organizational and technical mechanisms for continuous monitoring.
Harmonization of spatial data sets and services
While carrying out the activities regarding harmonization of spatial data sets and services, we have elaborated a strategy for harmonization of the Polish Spatial Data Infrastructure. The harmonization strategy of the Polish Spatial Data Infrastructure describes the activities that should be undertaken in order to adjust the national spatial data sets and services to the requirements described in the executory provisions of the INSPIRE Directive, 2007/2/WE as well as the Act of March 4th 2010 on the Spatial Data Infrastructure. The strategy describes the necessary activities, that are needed to transform the data sets, from their current form (the way they are currently collected and managed by the public administration organs of third parties) to a representation that will be compliant with the harmonized executory provisions. The assumptions that we elaborated while drafting the strategy, have been tested in the harmonization test laboratory. Subsequently, in line with the strategy, we have carried out harmonization of the spatial data sets of the Head Office of Geodesy and Cartography, using the newly developed harmonization tools.
The metadata, allowing for easier searching, obtaining and reusing spatial data, make another important element of the spatial data infrastructure. The metadata enable a potential user to assess in how far the spatial data sets available will be useful for his needs and to utilize these data in a specific application. According to the Act on Spatial Data Infrastructure as well as the Act on the Geodetic and Cartographic Law, all data sets of the spatial infrastructure should be described with metadata. The Head Office of Geodesy and Cartography has undertaken necessary organizational activities in the area of metadata creation, which included: preparation of assumptions for metadata elaboration, adjusting metadata profiles for spatial data resources as well as elaborating the guidelines for metadata creation.
Under the above mentioned activities , we have developed the metadata for spatial data sets. The metadata have been prepared in the XML format, in compliance with the assumptions elaborated earlier and with the usage of newly developed tools for edition, validation and automatic metadata generation.
Editor, Validator and Metadata Generator – these are the new tools, used for:
- creating metadata for data sets,
- creating series of data sets and services,
- mass development of metadata files, consistent with the metadata profile,
- mass validation of metadata files, according to the application scheme,
- export of files to a database with metadata,
- managing tags of metadata files.
The Metadata Editor is a tool, publically available, that allows to create metadata both in Polish and other languages (according to specific needs), compliant with ISO 19115, 19119 and 19139. The editor allows to create any metadata profiles, introducing metadata hierarchies, using dictionaries and thesaurus as well as generating a metadata file tag and automatic time stamp on the metadata. The editor is available online, via a web-based interface, API interface or as a web service.
The Metadata Validator allows to verify correctness of the metadata files, in terms of their compliance to ISO (19115, 19119, 19139)(, the INSPIRE profile and the metadata profile. It also allows for defining the requirements and verifying their consistence with new metadata profiles based on ISO. The validator allows both the person who created metadata and the catalogue administrator to assess correctness of the metadata xml file. The validation process can be run on metadata files as well as metadata collected in the metadata database on the Geoportal system catalogue server. The validator is available online, via a web-based interface, API interface or as a web service.
The application for automatic generation of metadata allows for automatic creation of metadata as well as conversion of the already created metadata for the spatial data resources, according to the previously developed metadata profile. The application is integrated with the metadata validator.
The tools for harmonization of data sets, tools for editing and validating the data sets as well as harmonized data sets, created within the above mentioned tasks, have been integrated into the Geoportal system. It is worth mentioning that the harmonized data sets allow for creation of services according to the INSPIRE requirements, with the help of tools available in the Geoportal system.
The Geoportal system is used as both the national and sector broker for the spatial data services as well as the national broker for INSPIRE services. Most elements of the system is common for all these roles of the Geoportal system. The sector broker is aimed at providing access to data transformation and collection services as well as being an online shop. The national broker is to provide dedicated services of spatial data collection, search, browsing and transformation. A similar role for spatial services and data consistent with the INSPIRE requirements, is played by the national broker for INSPIRE services. The Geoportal system is also used as a central node of the Spatial Data Infrastructure.
The Geoportal system acts as intermediary in the process of accessing spatial data services and infrastructure services. This role stems from the integrator model which has been used in the project, meaning that the system acts as an integrator, providing the consumers of spatial data with effective means, allowing for access to spatial data produced by different parties (spatial data producers). Thanks to the use of integrator model, the customer does not have to engage in the troublesome process of collecting and updating the data needed to use the services, which remain within the competence of multiple providers. The integrator role is served by an application service: the G2 service bus. The G2 service bus provides a homogeneous access point for all the customers using services of the Geoportal system.
Architecture of the INSPIRE, national and sector broker – data offering process
The activity regarding implementation and maintenance of the INSPIRE services included: performing a requirements analysis, developing the Geoportal system architecture, describing the architecture of catalogue services in Poland, according to the SOA paradigm, as well as developing supporting tools, such as tools to manage publishing data and services of the Geoportal system, tools to manage the content of Geoportal web site, tools to collect, manage and publish various data sets as well as monitoring and reporting tools.
Development of tools to manage publishing data and services of the Geoportal system included the following activities:
- enlarging the functionalities of the spatial data services server (including implementation of solutions that allow to publish services that are a result of the contract in question, services related to calibrating the spatial data services server as well as delivery of a software unit, allowing for management of the services offered),
- creating a scheme and a metadata database, according to a project, and loading it with metadata of data and services offered by the central node as well as metadata developed under the contract for harmonization,
- enlarging functionalities of the catalogue server (including transactions, harvesting and implementation of distributed queries, according to the CSW 2.0.2 specification as well as cooperation with the metadata database),
- delivering client software for metadata, integrated with the metadata editor and validator, allowing for management of the metadata catalogue contents.
A new information web site, developed under the Geoportal 2 project, is prepared to present newly launched functionalities and the new user interface has been designed to allow the weak-sighted to use the service. The searching and browsing functionalities have also been developed, making data browsing and searching much easier. In addition to that, we optimized the user interface and integrated it with an online shop. In order to support data collection, a data collection client has been developed, publically available and connected to Geoportal.
The tools developed under the project support collection, management and publishing of various data sets.
Dedicated tools for earial photos suport managing the aerial photos, import as well as automatic and semi-automatic classification and calibration of photos. We have also developed a database, including references to scanned photos from the CODGiK resource and developed metadata. A client software for aerial photos has been developed to allow for publishing aerial photos as well as accessing them via browsing and collecting services.
The functionalities of the orthophotomap management module have also been enlarged (including functionalities such as: import of multiple files, mass data transformation, data conversion, forcing updates of publishing services and managing orthophotomap statuses). The orthophotomap warehouse has been updated with references to all orthophotomaps. The orthophotomaps have also been offered via browsing and collecting services.
An warehouse collecting indexes of cartographic documents supplied by CODGiK in both digital and analogue format has been created. The warehouse has been loaded with existing indexes. We have also developed software enabling management of spreadsheet indexes, integrated with the application that manages the CODGiK resource and ensured access to data via a browsing service.
For NMT data, a dedicated warehouse has been created (including a scheme and a database with NMT data in 1:10 000 scale as well as NMT in raster, shaded and hypsometric form, created from the vector NMT data). The tools for NMT data allow to import NMT data from source formats and their automatic conversion into raster form, both shaded and hypsometric, as well as converting the data into a common data set, in a form required by the publication services. The services for browsing and collecting the NMT data have been made publically available. Data will be collected in GML files, consistent with the application scheme. The use of tiling technology allows for optimal browsing of the resources.
The tools delivered under the project allow to import data collected in the Control Network Database. Together with an activity aimed at creating a database with information on all control network points registered in the Control Network Bank, they will allow to offer services of data browsing and collecting to all interested parties. Data on the control network will be collected in form of GML files, consistent with the application scheme.
Development of tools to manage the Topographic Database (Topograficzna Baza Danych – TBD) will include integration with the National Management System of the Database of Topographic Objects (Baza Danych Obiektów Topograficznych – BDOT). The range of data accumulated in the TBD data warehouse will be enlarged, including the data uploaded from BDOT. The services for browsing and collecting TBD data have been offered to all interested parties. Data will be collected in form of GML files, consistent with the application scheme. The use of tiling technology allows for optimal browsing of the resources.
For the PRNG data set, there will be further work related to enlarging the data warehouse and converting source data to a coherent data set, in a form required by the data publishing services. The services of browsing and collecting PRNG data have been made available to all interested parties. Data will be collected in form of GML files, consistent with the application scheme.
For the PRG data set, services of browsing and collecting data will also be prepared and made available to all interested parties. Data will be collected in form of GML files, consistent with the application scheme. The data published will be suitable for generalization, based on source data.
The tools for collecting and publishing BDO developed under the project are based on metadata describing the BDO files and BDOO data tiles, generated in various coordinate systems. Services for browsing and collecting the BDO data will be prepared and made available to all interested parties. Data will be collected in form of GML files, consistent with the application scheme. The use of tiling technology allows for optimal browsing of the resources.
Services for searching, browsing and collecing data will also be prepared for the Vmap2 data set. These services will be made available to all interested parties. Data will be collected in form of GML files, consistent with the application scheme. The use of tiling technology allows for optimal browsing of the resources. The services will base on Vmap2 data tiles, generated in various coordinate systems.
In the area of publishing and managing the INSPIRE data, an INSPIRE data management module will be created and integrated with the data harmonization tools. We will also create an INSPIRE data warehouse, which will be loaded with data developed under the contract for data harmonization. There will be WMS, WFS, WCTS and WMTS services prepared for the INSPIRE data set and made available to all interested parties.
In order to support managing the CODGiK resource, an online shop has been created. This tool is integrated with the web client for browsing and searching for data as well as the data collection client. The software provides standard functionalities of an online shop (searching the stock with the use of attribute and spatial criteria, tracking order realization status, different payment methods, etc.) as well as functionalities specific to that type of stock (such as searching for stock fulfilling the criteria of analyses and spatial requests).
In the course of implementation, we have also developed tools for INSPIRE monitoring and reporting for both internal and external monitoring.
The SDI Module
The main task of the SDI Module is to store, manage and offer data and metadata that are at disposal of the Module user, with the use of spatial data services. The module does also allow for exchange of data with other nodes of the spatial data infrastructure (other SDI users – such as government or self-government units) and ensures synchronization of spatial data databases with the use of spatial data services.
Possible topological connections of the SDI Module
The local SDI nodes cooperate in the so-called star topology, i.e. they are connected to a single SDI Central Node.
Elements of a local SDI node
Elements of the SDI include:
A spatial data database – allowing to store spatial data sets, setting spatial tags on the data stored in order to allow quicker responses to queries as well as storing the data in a coordinates system chosen by the user;
A management system for the spatial database – allowing to execute administrative activities related to management of the spatial data database, like: managing database users and their rights, browsing and modifying data;
A tool for loading the SDO database – allowing to load the spatial database (with GML files or SWDE files) in two modes: initial mode (when the user loads the database for the first time) and incremental mode (actualization with new data). The tool does also allow to monitor all activities related to spatial data transformation, e.g. maintaining consecutive versions of transformed data;
The spatial data server – allowing to publish spatial data services with the use of:
- WMS services (Web Map Services) – representing geographical data as pictures;
- WFS services (Web Feature Services) – representing data as vectors, thus allowing for data collection and transformation;
- WMTS services (Web Map Tile Services) – representing geographic data as a picture, built up from a tile matrix;
Data for the Spatial data server are provided by the spatial data database. It is also able to use other sources, such as: Oracle databases, PostgreSQL databases as well as SHP, GML or GeoTIFF files.
The Catalogue services server – allowing to store, create and publish the metadata resource. The functionality of the catalogue services server is realized according to the CSW 2.0.2 specification (OpenGIS® Catalogue Services Specification). The Catalogue services server will be using such tools as metadata editor and validator, developed under the GEOPORTAL2 project and published by the Head Office of Geodesy and Cartography;
Tools to manage synchronization of the local nodes (SDI Module instances) – ensuring and facilitating management of data flow within the Spatial Data Infrastructure. Functionality of these tools allows to:
- divide local nodes into groups, attributing defined synchronization models to each group;
- define synchronization method: range of data to be subject to synchronization (data and their attributes), direction of data flow (from node/to node), data synchronization model.
The SDI module allows any entity that owns spatial data sets to offer standardized spatial data services, maintaining interoperability and standardizing access to data for the users of Spatial Data Infrastructure. An entity using the SDI Module becomes part of the Spatial Data Infrastructure and thus gains access to spatial data resources owned by the Head Office of Geodesy and Cartography and the ability to exchange data with other entities using the SDI Module.
The Universal Map Module (UMM) is made of a set of application tools that support the users' activities, e.g. by performing analyses on spatial data. The solutions adopted for UMM allow to apply a homogeneous architecture and use the following set of features in all modules:
- a common GUGiK service bus,
- common services for authentication and authorization,
- a common security policy,
- a homogeneous monitoring and reporting system,
- homogeneous access to spatial data, deposited in data warehouses of Geoportal.
The current solution of the UMM is dedicated to the rescue services – it has been developed upon requirements of the rescue services (including: handling of mobile appliances), however its functionality (especially advanced analytic methods) can also be used by other beneficiaries. There is a possibility to adapt the Module to individual requirements of its users.
Basic functionalities of the UMM include:
access to spatial data via SDI Module,
managing spatial data deposited in databases of the Universal Map Module, in the following areas:
- spatial data visualization,
- searching for spatial objects with help of defined parameters,
- spatial data analyses (e.g. displaying a place, where defined events occur, analyzing intensity of events),
- analyzing statistical data (e.g. trend analysis, forecasts),
- network analyses (e.g. tracing access routes, tracing the closest localization of objects),
- import/export of data,
preparation of cartographic printouts (maps),
preparation and generation of reports.
The UMM uses the referential spatial data owned by the Head Office of Geodesy and Cartography as well as geospatial data from entities using the Universal Map Module. UMM can use the SDI Module solutions in the area of management and access to spatial data.
UMM utilisation model
A UMM user has access to UMM application tools, that use referential spatial data from GUGiS as well as geospatial data deposited by users of the SDI Module. It means that the more spatial data are deposited in the data warehouse of the SDI central node, the larger become the analytic capabilities of the Universal Map Module.
Technical infrastructure of the geoportal
The present technical infrastructure of the Geoportal system has been based on infrastructure delivered within the geoportal.gov.pl project. The activities concerning the technical infrastructure within the Geoportal 2 project were focused on maintaining the existing infrastructure, optimizing its usage and sustainable development. The existing infrastructure of the Geoportal system is maintained according to procedures, based on ITIL best practices.
Developement of the technical infrastructure, aimed at adjusting the existing infrastructure to the needs of the Geoportal 2 project, has been based on sizing the Geoportal 2 services. The method used allowed to determine results regarding the number of business operations and their distribution according to business services, data volumes and their distribution according to business services as well as indicators presenting efficiency of both application and database layers, expressed in units that are an industry standard. Results of sizing allowed for optimal planning of infrastructure development.
Development of infrastructure included the following activities:
- providing sufficient capacity of Internet access ((primary and reserve connection),
- adding licenses for the virtualization environment as well as for standard software for servers, application servers, databases and operating systems as well as other software,
- developing the network infrastructure,
- creating mechanisms for infrastructure monitoring,
- developing infrastructure that provides system security,
- developing hardware infrastructure.
Using the consolidation platform in the database layer and storage space proved to be an important factor optimizing the infrastructure usage. This solution ensured high quality of the environment, which was also achieved with the use of a smaller amount of database processes and other hardware resources, than would be needed in case of traditional architecture. The solution did also prove to generate lower costs, which was due to optimization of a number of core database processors, lower needs for computational power of the servers as well as easier and less labor-intensive management of the IT environment. The usage of a consolidation platform does also allow for a fast process of identifying and reproducing any eventual problems in an integrated environment, covered with a homogeneous supporting service and with an ability to flexibly and automatically distribute resources of the database platform, including computational power and disc capacity, according to current needs.