Migrate Data Between GeoNode Instances¶

This workshop shows how to migrate GeoNode Layers (along with GeoServer associated datasets and styles) from an instance to another.

The whole tutorial is divided in different parts, each one showing a different methodology to perform the data migration:

Manual migration of data between to GeoNode instances with same version
Semi-automatic migration of data between GeoNode instances with same version
Automatic migration of data between GeoNode instances with same version
Automatic migration of data between GeoNode instances with different version

Warning

Currently the points 2,3,4 are not yet feasable with the actual GeoNode version. Those section will be available as soon as the “GeoNode Backup & Restore” GNIP development will be ready and merged to the master branch.

Manual migration of data between GeoNode instances with same version¶

This section shows how to export a Layer from a GeoNode instance and import it back to another one with the same versions.

Before going through the commands and the operations to perform the export/import tasks, the tutorial will explain in details the structure of a GeoNode Layer.

As you may already know, the physical geospatial data, along with its graphical stylesheets (also known as SLDs), are backed by GeoServer. Each GeoNode version is shipped with a related GeoServer version and GeoServer Data Dir (we will see in details later what does it means). For the moment the important thing to know is that GeoNode cannot live without a running instance of GeoServer. Therefore migrating data from a GeoNode instance to another one, means also move geospatial data and stylesheets between the related GeoServer instances.

A GeoNode Layer Structure¶

Lets start anlysing deeply a GeoNode Layer structure.

Preparation Of The Webinar¶

This tutorial is generic and can be executed using any existing Layer of GeoNode, however in order to follow exactly the same passages, please execute this first simple exercise.

Warning

As a prerequisite, you must have access to a GeoNode instance with Admin rights.

Exercise

Add A Sample Layer To GeoNode

Log into GeoNode as Administrator
Click on the Add Layers button from the home page, in order to switch to the upload page

GeoNode Add Layers Button
Click on the Browse on the upload page

GeoNode Upload Browse Button
Select from the folder gisdata/data/good/vector the 4 files
- san_andres_y_providencia_coastline.dbf
- san_andres_y_providencia_coastline.prj
- san_andres_y_providencia_coastline.shp
- san_andres_y_providencia_coastline.shx
GeoNode Upload Browse Button
Click on the Upload Files button and make sure the operation completes successfully

GeoNode Upload Files Button

Layer Metadata¶

Each resource in GeoNode has metadata. Metadada in GeoNode is quite important, it is used by the application to describe, search and identify a resource. As an instance part the title, abstract, regions or keywords of a resource are all part of the metadata.

Metadata in GeoNode is stored on the backend databse as a set of fields associated to the resource. The catalogue service then, makes use of such information to dynamically generate ISO compliant XML records. Those records can be used by external catalogiung applications, compliant with the standards supported by GeoNode, in order to automatically recognize and indexing the available resources on the server.

The GeoNode Layer info page makes use of some metadata fields to provide immediate description of the Layer.

As shown in the figure below, the GeoNode Info tab contains a table reporting basic information about the Layer, like the Title, the Abstract, the Category and others.

GeoNode Layer Info

Warning

The Title shown on the Layer Metadata is not the real Layer name. We will deepen the topic on the following sections of this tutorial.

In order to obtain the whole Layer Metadata in one standard format (usually not quite human-friendly; a huge and long XML), it is possible to click on the Download Metadada button on the right panel. GeoNode will present to the user a modal window with a list of permalinks to the dynamic XML supported formats.

GeoNode Layer Download Metadada

As an instance, if you click on ISO, you will get an XML containing all the Layer metadata fields in ISOTC211/19115 format.

GeoNode Layer Download Metadada ISOTC211/19115 Format

Excercise

Edit The Layer Metadada

Go to the GeoNode Layer List

GeoNode Layer List
Click on a Layer in order to go to the resource info page

GeoNode Layer Edit
Click on the Edit Layer button

GeoNode Layer Download Metadada Button
Click on the Edit button under the Metadada incon of the modal window

GeoNode Metadada Edit
Update the at least the Title, the Abstract and the Category and finally click on the Update button

GeoNode Metadada Edit

The export of metadata is a foundamental task to achieve when moving a resource from a GeoNode instance to another.

Excercise

Export the Layer Metadata as ISOTC211/19115 and save it to an XML files on the local storage

Go to the GeoNode Layer List

GeoNode Layer List
Click on a Layer in order to go to the resource info page

GeoNode Layer Edit
Click on the Download Metadada button

GeoNode Layer Download Metadada Button
From the modal window, click with the right mouse button over the ISO link

GeoNode Layer Download Metadada ISOTC211/19115 Format
From the context menù, select the voice Save Link As

GeoNode Layer Download Metadada ISOTC211/19115 Save Link As
Store the xml into the hard disk and note the location for later use

GeoNode Layer Download Metadada ISOTC211/19115 XML

Layer Styles¶

Each Layer in GeoNode has a representation style associated, or a Legend if you want.

A style is basically a set of rules instructing the geospatial server on how to create a portrayal of the original data. The figure shown in the map is only one of the inifite possible representation of the data stored on the server.

It is worth to point out that viewing the data is substantially different from getting the data. A portrayal of data provides to the users an immediate understanding of the meaning (or at least of one possible meaning), but this is not suitable for data analysis or more sophisticated computational tasks.

The legend (or style), depends exclusvely from the geometry of the layer and, optionally, from a subset of its atributes.

On GeoNode, if you move to a Layer info page, you can notice a small Legend panel on the right representing the style currently in use.

GeoNode Layer Download Metadada ISOTC211/19115 XML

A Layer can have a lot of different styles associated, of course. Usually there is a Default Style which is the one presented to the users if not differently specified.

It is possible from the GeoNode interface to manage the styles associated to a Layer and also change its Default Style.

Note

Only owners or users with write permissions on the Layer can update the styles.

Excercise

Layer Styles Management Panel

Go to the GeoNode Layer List

GeoNode Layer List
Click on a Layer in order to go to the resource info page

GeoNode Layer Edit
Click on the Edit Layer button on the right panel

GeoNode Layer Edit Button
Click on the Manage button under the Styles incon of the modal window

GeoNode Layer Styles Manage
Play with the styles comboboxex in order to change the Default Style or add/remove more of them without updating the Layer

GeoNode Layer Styles Management Panel

Warning

Do not click on Update Available Styles button, otherwise you will change the current Layer styles.

GeoNode also provides a simple tool for the editing of the Layer style directly from the web interface.

Note

It is worth noting that the GeoNode style editor is very simple and does not allow advanced style editing. Also this tool may not work perfectly with complex layers. Further in the tutorial we will see how it is possible to edit directly the style using the SLD native format.

Excercise

Update the default style through the GeoNode Style Editor tool

Go to the GeoNode Layer List

GeoNode Layer List
Click on a Layer in order to go to the resource info page

GeoNode Layer Edit
Click on the Edit Layer button on the right panel

GeoNode Layer Edit Button
Click on the Edit button under the Styles incon of the modal window

GeoNode Layer Styles Edit
You should see a small window similar to the one depicted below

GeoNode Layer Styles Editor
Select the first Rule and click on the small Edit button below

GeoNode Layer Styles Edit Rules
Modify the Name, the Color and the Width of the stroke and click save

GeoNode Layer Styles Edit Stroke

The GeoNode style editor tool just simplifies the editing of a Layer style by providing a small graphic user interface to one of the GeoServer capabilities.

Under the hood a Layer style is a special XML format defined from the Open Geospatial Consortium (OGC) as Style Layer Descriptor or SLD.

Advanced users can directly modify the SLD or use more advanced tools to create very complex and beautiful Layer styles.

In order to do that, you will need to update the SLD source directly through the GeoServer interface.

Excercise

Update the default style through the GeoServer interface

Log inot GeoNode as Administrator. Then click on the user button on the top right.

GeoNode Admin
From the menu, click on the GeoServer voice.

GeoNode Admin GeoServer
You will be redirected to the GeoServer admin interface.

GeoServer Admin Gui
Select the Styles topic from the left menu.

GeoServer Admin Styles
Select the layer name from the list and click on it. You will be redirected to the SLD editor page.

GeoServer Admin Style Editor
Modify the Color and the Width of the External Border XML rule. Click on Preview legend to see the changes, and when you are happy Submit the SLD.

GeoServer Admin Style Editor
Go back to GeoNode. Reload the Layer in order to see the changes.

GeoNode Updated Layer Style

GeoServer Web Interface Basics¶

This section will introduce the basic concepts of the web administration interface (generally abbreviated to “web admin” .)

Welcome Page

For most installations, GeoServer will start a web server on localhost at port 8080, accessible at the following URL:

http://localhost:8080/geoserver/web

Note

This URL is dependent on your installation of GeoServer. When using the WAR installation, for example, the URL will be dependent on your container setup. A GeoNode/GeoServer production environment, usually maps GeoServer un the same port 80 like the following

http://<geonode_host>/geoserver/web

When correctly configured, a welcome page will open in your browser.

Welcome Page

The welcome page contains links to various areas of the GeoServer configuration. The About GeoServer section in the Server menu provides external links to the GeoServer documentation, homepage, and bug tracker. The page also provides login access to the geoserver console. This security measure prevents unauthorized users from making changes to your GeoServer configuration. The default username and password is admin and geoserver. These can be changed only by editing the security/users.properties file in the data_directory.

Login

Regardless of authorization access, the web admin menu links to the Demo and Layer Preview portion of the console. The webadmin_demos page contains links to various information pages, while the layerpreview page provides spatial data in various output formats.

Warning

On a GeoNode/GeoServer environment, you need to login on GeoNode first. The authorization is automatically taken by GeoServer through your browser cookies.

Once logged into GeoNode, going to http://<geonode_host>/geoserver/web will automatically log into GeoServer with the same rights.

Note

On a GeoNode/GeoServer environment, GeoServer still allows FORM and BASIC authentication mechanism (like shown in this section). But this is useful only if you need to access to GeoServer from external applications or with another/specific GeoServer user. You must know the username and passowrd of the GeoServer user in this case.

When logged on, additional options will be presented.

Additional options when logged in

Geoserver Web Coverage Service (WCS), Web Feature Service (WFS), and Web Map Service (WMS) configuration specifications can be accessed from this welcome page as well.

List Pages

Some web admin pages show list views of configuration data type items available in the GeoServer instance. The page displays links to the items, and where applicable their parent items as well. To facilitate working with large sets of items, list views allow sorting and searching across all items in the data type.

In the example below, the GeoServer Layers page displays a list of layers along with links to their parent GeoServer Stores and GeoServer Workspaces.

Layers list page

Sorting

To sort a column alphabetically, click the column header.

Unsorted (left) and sorted (right) columns

Searching

Searching can be used to filter the number of items displayed. This is useful for working with data types that contain a large number of items.

To search data type items, enter the search string in the search box and click Enter. GeoServer will search the data type for items that match your query, and display a list view showing the search results.

Search results for the query “coast” on the Layers page

GeoServer Workspaces¶

This section describes how to view and configure workspaces. Analogous to a namespace, a workspace is a container which organizes other items. In GeoServer, a workspace is often used to group similar layers together. Layers may be referred to by their workspace name, colon, layer name (for example geonode:san_andres_y_providencia_coastline). Two different layers can have the same name as long as they belong to different workspaces (for example sf:states and topp:states).

Note

On a GeoNode/GeoServer environment, by default there is only one workspace defined called geonode. You can still define other workspaces, but GeoNode will work only with Layers under the geonode workspace.

Workspaces page

Edit a Workspace

To view or edit a workspace, click the workspace name. A workspace configuration page will be displayed.

Workspace named “geonode”

A workspace is defined by a name and a Namespace URI (Uniform Resource Identifier). The workspace name is limited to ten characters and may not contain space. A URI is similar to a URL, except URIs do not need to point to a actual location on the web, and only need to be a unique identifier. For a Workspace URI, we recommend using a URL associated with your project, with perhaps a different trailing identifier. For example, http://www.geonode.org/ is the URI for the “geonode” workspace, http://www.openplans.org/topp is the URI for the “topp” workspace.

Add a Workspace

The buttons for adding and removing a workspace can be found at the top of the Workspaces view page.

../../_images/data_workspaces_add_remove.png

Buttons to add and remove

To add a workspace, select the Add new workspace button. You will be prompted to enter the the workspace name and URI.

../../_images/data_workspaces_medford.png

New Workspace page with example

Remove a Workspace

To remove a workspace, select it by clicking the checkbox next to the workspace. Multiple workspaces can be selected, or all can be selected by clicking the checkbox in the header. Click the Remove selected workspaces(s) button. You will be asked to confirm or cancel the removal. Clicking OK removes the selected workspace(s).

../../_images/data_workspaces_rename_confirm.png

Workspace removal confirmation

Warning

Removing a workspace will delete also all the layers associated with it.

Workspaces On The GeoServer Data Dir

All the configuration files of a workspace are stored into the GeoServer Data Dir. We will see later, on another section of the tutorial, how to access this folder.

It is worth noting that there is also a physical dependency between workspaces, stores and layers.

A GeoServer layer is always defined by its workspace and store. In the GeoServer Data Dir the layer definition is stored as a subdirectory of its store, the store definition is stored as a subdirectory of its workspace.

 <data_directory>/

    ...

    workspaces/
      |
      +- workspace dirs...
         |
         +- datastore dirs...
            |
            +- layer dirs...

GeoServer Stores¶

A store connects to a data source that contains raster or vector data. A data source can be a file or group of files, a table in a database, a single raster file, or a directory (for example, a Vector Product Format library). The store construct allows connection parameters to be defined once, rather than for each dataset in a source. As such, it is necessary to register a store before configuring datasets within it.

Stores View

Store types

While there are many potential formats for data sources, there are only four kinds of stores. For raster data, a store can be a file. For vector data, a store can be a file, database, or server.

Type Icon	Description
	raster data in a file
	vector data in a file
	vector data in a database
	vector server (web feature server)

Edit a Store

To view or edit a store, click the store name. A store configuration page will be displayed. The exact contents of this page depend on the specific format of the store. See the sections Working with Vector Data, Working with Raster Data, and Working with Databases for information about specific data formats. The example shows the configuration for the nurc:ArcGridSample store.

Editing a raster data store

Basic Store Info

The basic information is common for all formats.

Workspace - the store is assigned to the selected workspace
Data Source Name - the store name as listed on the view page
Description - (optional) a description that displays in the administration interface
Enabled - enables or disables access to the store, along with all datasets defined for it

Connection Parameters

The connection parameters vary depending on data format.

Add a Store

The buttons for adding and removing a store can be found at the top of the Stores page.

../../_images/data_stores_add_remove.png

Buttons to add and remove a Store

To add a store, select the Add new Store button. You will be prompted to choose a data source. GeoServer natively supports many formats (with more available via extensions). Click the appropriate data source to continue.

Choosing the data source for a new store

The next page configures the store. Since connection parameters differ across data sources, the exact contents of this page depend on the store’s specific format. See the sections data_vector, data_raster, and data_database for information on specific data formats. The example below shows the ArcGrid raster configuration page.

Configuration page for an ArcGrid raster data source

Remove a Store

To remove a store, click the checkbox next to the store. Multiple stores can be selected, or all can be selected by clicking the checkbox in the header.

Stores selected for removal

Click the Remove selected Stores button. You will be asked to confirm the removal of the configuration for the store(s) and all resources defined under them. Clicking OK removes the selected store(s), and returns to the Stores page.

../../_images/data_stores_delete_confirm.png

Confirm removal of stores

How GeoNode Automatically Configures Workspaces And Stores¶

GeoNode currently creates the Stores automatically on Layer upload.

Summarizing GeoNode does:

GeoNode uses always the Workspace geonode for each Layer
GeoNode configures automatically the Stores for the Layers
- For Raster Layers; GeoNode creates a Store of type GeoTIFF with the same name of the Layer
- For Vectorial Layers; GeoNode creates a Store of type ESRI Shapefile with the same name of the Layer only if not connected to a Database
- For Vectorial Layers; GeoNode uses always the same Store of type Postgis and convert the vectorial data into Database Tables

Excercise

GeoNode Workspace And Stores

Log inot GeoNode as Administrator. Then click on the user button on the top right.

GeoNode Admin
From the menu, click on the GeoServer voice.

GeoNode Admin GeoServer
You will be redirected to the GeoServer admin interface.

GeoServer Admin Gui
Select the Workspaces topic from the left menu.

GeoServer Workspace Menu
Select the geonode workspace from the list.

GeoServer Workspace List
Edit the geonode workspace.

GeoServer Workspace geonode
Select the Stores topic from the left menu.

GeoServer Store Menu
If GeoNode and GeoServer have been configured to use a Database backend, select the store datastore of type PostGIS.

GeoServer Store List

GeoServer Store datastore

Note

The Datastore is confiugured by using the same connection parameters specified in the GeoNode local_settings.py file.
If GeoNode and GeoServer have been configured to use Shapefiles, select the store san_andres_y_providencia_coastline of type Shapefile.

GeoServer Store List

GeoServer Store san_andres_y_providencia_coastline

Note

The Store is confiugured by pointing directly to the path of the san_andres_y_providencia_coastline.shp file.

GeoServer Layers¶

In GeoServer, the term “layer” refers to a raster or vector dataset that represents a collection of geographic features. Vector layers are analogous to “featureTypes” and raster layers are analogous to “coverages”. All layers have a source of data, known as a Store. The layer is associated with the Workspace in which the Store is defined.

In the Layers section of the web interface, you can view and edit existing layers, add (register) a new layer, or remove (unregister) a layer. The Layers View page displays the list of layers, and the Store and Workspace in which each layer is contained. The View page also displays the layer’s status and native SRS.

Layers View

Layer types

Layers can be divided into two types of data: raster and vector. These two formats differ in how they store spatial information. Vector types store information about feature types as mathematical paths—a point as a single x,y coordinate, lines as a series of x,y coordinates, and polygons as a series of x,y coordinates that start and end on the same place. Raster format data is a cell-based representation of features on the earth surface. Each cell has a distinct value, and all cells with the same value represent a specific feature.

Field	Description
	raster (grid)
	vector (feature)

Add a Layer

At the upper left-hand corner of the layers view page there are two buttons for the adding and removal of layers. The green plus button allows you to add a new layer (referred to as resource). The red minus button allows you to remove selected layers.

../../_images/data_layers_add_remove.png

Buttons to Add and Remove a Layer

Clicking the Add a new resource button brings up a New Layer Chooser panel. The menu displays all currently enabled stores. From this menu, select the Store where the layer should be added.

../../_images/data_layers_add_chooser.png

List of all currently enabled stores

Upon selection of a Store, a list is displayed of resources within the store. Resources which have already been published as layers are listed first, followed by other resources which are available to be published. In this example, giant_polygon, poi, poly_landmarks and tiger_roads are all existing layers within the NYC store.

List of published and available resources in a store

To add a layer for an available resource click Publish. To add a new layer for a published resource click Publish Again. (Note that when re-publishing the name of the new layer may have to be modified to avoid conflict with an existing layer.) The actions display an Edit Layer page to enter the definition of the new layer.

Remove a Layer

To remove a layer, select it by clicking the checkbox next to the layer. As shown below, multiple layers can be selected for batch removal. Note that selections for removal will not persist from one results pages to the next.

Some layers selected for removal

All layers can be selected for removal by clicking the checkbox in the header.

../../_images/data_layers_delete_all.png

All layers selected for removal

Once layer(s) are selected, the Remove selected resources link is activated. Once you’ve clicked the link, you will be asked to confirm or cancel the removal. Selecting OK removes the selected layer(s).

Edit Layer: Data

To view or edit a layer, click the layer name. A layer configuration page will be displayed. The Data tab, activated by default, allows you to define and change data parameters for a layer.

Edit Layer: Data tab

Basic Info

The beginning sections—Basic Resource Info, Keywords and Metadata link—are analogous to the Service Metadata section for WCS, WFS, and WMS. These sections provide “data about the data,” specifically textual information that make the layer data easier to understand and work with. The metadata information will appear in the capabilities documents which refer to the layer.

Name—Identifier used to reference the layer in WMS requests. (Note that for a new layer for an already-published resource, the name must be changed to avoid conflict.)
Title—Human-readable description to briefly identify the layer to clients (required)
Abstract—Describes the layer in detail
Keywords—List of short words associated with the layer to assist catalog searching
Metadata Links—Allows linking to external documents that describe the data layer. Currently only two standard format types are valid: TC211 and FGDC. TC211 refers to the metadata structure established by the ISO Technical Committee for Geographic Information/Geomatics (ISO/TC 211) while FGDC refers to those set out by the Federal Geographic Data Committee (FGDC) of the United States.

Adding a metadata link in FGDC format

Coordinate Reference Systems

A coordinate reference system (CRS) defines how georeferenced spatial data relates to real locations on the Earth’s surface. CRSes are part of a more general model called Spatial Reference Systems (SRS), which includes referencing by coordinates and geographic identifiers. GeoServer needs to know the Coordinate Reference System of your data. This information is used for computing the latitude/longitude bounding box and reprojecting the data during both WMS and WFS requests.

Coordinate reference system of a layer

Native SRS—Specifies the coordinate system the layer is stored in. Clicking the projection link displays a description of the SRS.
Declared SRS—Specifies the coordinate system GeoServer publishes to clients
SRS Handling—Determines how GeoServer should handle projection when the two SRSes differ

Bounding Boxes

The bounding box determines the extent of the data within a layer.

Native Bounding Box—The bounds of the data specified in the Native SRS. These bounds can be generated by clicking the Compute from data button.
Lat/Lon Bounding Box—The bounds specified in geographic coordinates. These bounds can be calculated by clicking the Compute from native bounds button.

Bounding Boxes of a layer

Feature Type Details (Vector)

Vector layers have a list of the Feature Type Details. These include the Property and Type of a data source. For example, the sf:archsites layer shown below includes a geometry (the_geom) of type “point”.

Feature Type Details

The Nillable option refers to whether the property requires a value or may be flagged as being null. Meanwhile Min/Max Occurrences refers to how many values a field is allowed to have. Currently both Nillable and Min/Max Occurrences are set to true and 0/1 but may be extended with future work on complex features.

Edit Layer: Publishing

The Publishing tab configures HTTP and WMS/WFS/WCS settings.

../../_images/data_layers_edit_publish.png

Edit Layer: Publishing tab

Enabled—A layer that is not enabled won’t be available to any kind of request, it will just show up in the configuration (and in REST config)
Advertised—A layer is advertised by default. A non-advertised layer will be available in all data access requests (for example, WMS GetMap, WMS GetFeature) but won’t appear in any capabilities document or in the layer preview.

HTTP Settings

Cache parameters that apply to the HTTP response from client requests.

Response Cache Headers— If selected, GeoServer will not request the same tile twice within the time specified in Cache Time. One hour measured in seconds (3600), is the default value for Cache Time.

WMS Settings

Sets the WMS specific publishing parameters.

WMS Settings

Queryable—Controls whether the layer is queryable via WMS GetFeatureInfo requests.
Default style—Style that will be used when the client does not specify a named style in GetMap requests.
Additional styles—Other styles that can be associated with this layer. Some clients (and the GeoServer Layer Preview) will present those as styling alternatives for that layer to the user.
Default rendering buffer—Default value of the buffer GetMap/GetFeatureInfo vendor parameter. See the Wms Vendor Parameters for more details.
Default WMS path—Location of the layer in the WMS capabilities layer tree. Useful for building non-opaque layer groups

WMS Attribution

Sets publishing information about data providers.

WMS Attribution

Attribution Text—Human-readable text describing the data provider. This might be used as the text for a hyperlink to the data provider’s web site.
Attribution Link—URL to the data provider’s website.
Logo URL—URL to an image that serves as a logo for the data provider.
Logo Content Type, Width, and Height—These fields provide information about the logo image that clients may use to assist with layout. GeoServer will auto-detect these values if you click the Auto-detect image size and type link at the bottom of the section. The text, link, and URL are each advertised in the WMS Capabilities document if they are provided. Some WMS clients will display this information to advise users which providers provide a particular dataset. If you omit some of the fields, those that are provided will be published and those that are not will be omitted from the Capabilities document.

WFS Settings

Per-Request Feature Limit—Sets the maximum number of features for a layer a WFS GetFeature operation should generate (regardless of the actual number of query hits)
Maximum number of decimals—Sets the maximum number of decimals in GML output.

Note

It is also possible to override the OtherSRS/OtherCRS list configured in the WFS service, including overriding it with an empty list if need be. The input area will accept a comma separated list of EPSG codes:

WFS otherSRS/otherCRS override

The list will be used only for the capabilities document generation, but will not be used to limit the actual target SRS usage in GetFeature requests.

How GeoNode Automatically Configures Layers¶

GeoNode automatically sets and updates information on GeoServer every time a Layer is created or its metadata updated.

Excercise

Verify Information On GeoServer Side

Log inot GeoNode as Administrator. Then click on the user button on the top right.

GeoNode Admin
From the menu, click on the GeoServer voice.

GeoNode Admin GeoServer
You will be redirected to the GeoServer admin interface.

GeoServer Admin Gui
Select the Layers topic from the left menu.

GeoServer Layers Menu
Select the Layer san_andres_y_providencia_coastline from the list.

GeoServer Layers List
Notice how the Basic Resource Info have been automatically configured by GeoNode.
- The Name reflects the original file name.
- The Title has been filled with the Metadata value
- The Abstract has been filled with the Metadata value
- The Keywords have been added with the Metadata values
GeoServer Layers Basic Resource Info
Notice how the Metadata Links reflect the Metadada Download options available on GeoNode, along with the URLs.

GeoServer Layers Metadata Links
Notice how the Coordinate Reference System and the Bounding Boxes have been automatically filled by GeoNode accordingly with the data source info.

GeoServer Layers Coordinate Reference System
On the Publishing Section the Default Style has been updated and configured by GeoNode.

GeoServer Layers Publishing Section

GeoServer Stylesheets (SLD)¶

Styles render, or make available, geospatial data. Styles for GeoServer are written in Styled Layer Descriptor (SLD), a subset of XML. Please see the section on Styling for more information on working with styles.

On the Styles page, you can add a new style, view or edit an existing style, or remove a style.

Styles page

Edit a Style

To view or edit a style, click the style name. A Style Editor page will be diplayed. The page presents options for configuring a style’s name, code, and other attributes. Style names are specified at the top in the name field. The style’s workspace can be chosen using workspace selector. Styles are edited using a plain text editor with some basic utilities.

Style editor

The style editor supports line numbering, automatic indentation, and real-time syntax highlighting. You can also increase or decrease the font size of the editor.

Button	Description
	undo
	redo
	go to line
	auto-format the editor contents
	change the font size of the editor

To confirm that the SLD code is fully compliant with the SLD schema, click the Validate button. A message box will confirm whether the style contains validation errors.

Note

GeoServer will sometimes render styles that fail validation, but this is not recommended.

../../_images/data_style_editor_noerrors.png

No validation errors

../../_images/data_style_editor_error.png

Validation error message

To view the generated legend entry for the style, click the Preview Legend button.

Add a Style

The buttons for adding and removing a style can be found at the top of the Styles page.

Adding or removing a style

To add a new style, select the Add a new style button. You will be redirected to an editor page. Enter a name for the style. You can also select the style format. In a default GeoServer installation only SLD is supported, but other extensions (such as css) add support for additional formats. The editor page provides several options for submitting a new style. You can paste the style directly into the editor contents. You can generate a new default style based on an internal template:

../../_images/data_style_editor_generate.png

Generating a new default style.

You can copy the contents of an existing style into the editor:

../../_images/data_style_editor_copy.png

Copying an existing Style from GeoServer

You can select and upload a local file that contains the SLD:

Uploading an SLD file from your local computer

Once a style is successfully submitted, you will be redirected to the main Styles page where the new style will be listed.

Remove a Style

To remove a style, select it by clicking the checkbox next to the style. Multiple styles can be selected, or all can be selected by clicking the checkbox in the header. Click the Remove selected style(s) link at the top of the page. You will be asked to confirm or cancel the removal. Clicking OK removes the selected style(s).

Confirmation prompt for removing styles

How GeoNode Automatically Configures Style¶

Whe uploading a new Layer to GeoNode, it creates by default an SLD on GeoServer with the same native name of the Layer and assigns it to the Layer ad Default Style.

In the previous sections you already modified the default style of the Layer san_andres_y_providencia_coastline automatically generated by GeoNode.

GeoServer Data Dir Structure¶

This section gives an overview of the structure and contents of the GeoServer data directory.

This is not intended to be a complete reference to the GeoServer configuration information, since generally the data directory configuration files should not be accessed directly. Instead, the Web Administration Interface can be used to view and modify the configuration manually, and for programmatic access and manipulation the REST configuration API should be used.

The directories that do contain user-modifiable content are: logs, palettes, templates, user-projection, and www.

The following figure shows the structure of the GeoServer data directory:

 <data_directory>/

    global.xml
    logging.xml
    wms.xml
    wfs.xml
    wcs.xml

    data/
    demo/
    geosearch/
    gwc/
    layergroups/
    logs/
    palettes/
    plugIns/
    security/
    styles/
    templates/
    user_projections/
    workspaces/
      |
      +- workspace dirs...
         |
         +- datastore dirs...
            |
            +- layer dirs...
    www/

The .xml files

The top-level .xml files contain information about the services and various global options for the server instance.

File	Description
`global.xml`	Contains settings common to all services, such as contact information, JAI settings, character sets and verbosity.
`logging.xml`	Specifies logging parameters, such as logging level, logfile location, and whether to log to stdout.
`wcs.xml`	Contains the service metadata and various settings for the WCS service.
`wfs.xml`	Contains the service metadata and various settings for the WFS service.
`wms.xml`	Contains the service metadata and various settings for the WMS service.

workspaces

The workspaces directory contain metadata about the layers published by GeoServer. It contains a directory for each defined workspace. Each workspace directory contains directories for the datastores defined in it. Each datastore directory contains directories for the layers defined for the datastore. Each layer directory contains a layer.xml file, and either a coverage.xml or a featuretype.xml file depending on whether the layer represents a raster or vector dataset.

data

The data directory can be used to store file-based geospatial datasets being served as layers. (This should not be confused with the main “GeoServer data directory”.) This directory is commonly used to store shapefiles and raster files, but can be used for any data that is file-based.

The main benefit of storing data files under the data directory is portability. Consider a shapefile stored external to the data directory at a location C:\gis_data\foo.shp. The datastore entry in catalog.xml for this shapefile would look like the following:

 <datastore id="foo_shapefile">
    <connectionParams>
      <parameter name="url" value="file://C:/gis_data/foo.shp" />
    </connectionParams>
  </datastore>

Now consider trying to port this data directory to another host running GeoServer. The location C:\gis_data\foo.shp probably does not exist on the second host. So either the file must be copied to this location on the new host, or catalog.xml must be changed to reflect a new location.

This problem can be avoided by storing foo.shp in the data directory. In this case the datastore entry in catalog.xml becomes:

  <datastore id="foo_shapefile">
    <connectionParams>
      <parameter name="url" value="file:data/foo.shp"/>
    </connectionParams>
  </datastore>

The value attribute is rewritten to be relative to the data directory. This location independence allows the entire data directory to be copied to a new host and used directly with no additional changes.

demo

The demo directory contains files which define the sample requests available in the Sample Request Tool (http://localhost/geoserver/demoRequest.do). See the Demos page for more information.

geosearch

The geosearch directory contains information for regionation of KML files.

gwc

The gwc directory holds the cache created by the embedded GeoWebCache service.

layergroups

The layergroups directory contains configuration information for the defined layergroups.

logs

The logs directory contains configuration information for logging profiles, and the default geoserver.log log file. See also Advanced log configuration.

palettes

The palettes directory is used to store pre-computed Image Palettes. Image palettes are used by the GeoServer WMS as way to reduce the size of produced images while maintaining image quality. See also Paletted Images.

security

The security directory contains the files used to configure the GeoServer security subsystem. This includes a set of property files which define access roles, along with the services and data each role is authorized to access. See the Security section for more information.

styles

The styles directory contains Styled Layer Descriptor (SLD) files which contain styling information used by the GeoServer WMS. For each file in this directory there is a corresponding entry in catalog.xml:

  <style id="point_style" file="default_point.sld"/>

See the Styling section for more information about styling and SLD .

templates

The templates directory contains files used by the GeoServer templating subsystem. Templates are used to customize the output of various GeoServer operations. See also Freemarker Templates.

user_projections

The user_projections directory contains a file called epsg.properties which is used to define custom spatial reference systems that are not part of the official EPSG database. See also Custom CRS Definitions.

www

The www directory is used to allow GeoServer to serve files like a regular web server. The contents of this directory are served at http:/<host:port>/geoserver/www. While not a replacement for a full blown web server, this can be useful for serving client-side mapping applications. See also Serving Static Files.

Excercise

Navigate the GeoServer Data Directory

Log inot GeoNode as Administrator. Then click on the user button on the top right.

GeoNode Admin
From the menu, click on the GeoServer voice.

GeoNode Admin GeoServer
You will be redirected to the GeoServer admin interface.

GeoServer Admin Gui
Select the Server Status topic from the left menu. On the status page note the Data Directory.

GeoServer Admin Server Status Page
Open a Terminal window and go to the GeoServer Data Directory folder. Navigate the folders and examine the files.
Enter the styles directory and confirm it is present the file san_andres_y_providencia_coastline.sld
1
$ cat san_andres_y_providencia_coastline.sld

Steps To Manually Migrate A Layer¶

In this section we are going to save on the mass storage all the information from a GeoNode Layer needed to restore it back later on another GeoNode instance.

Here below a quick summary of the actions to do in order to export manually a GeoNode Layer:

First of all it is necessary to store the Layer data in a portable format. Since we can’t assume any specific configuration on the target GeoNode (we don’t know if it has been attached to a Database or not, and the connection parameters neither), we need to save the Layer source data in a format feasable to be imported back on any GeoNode. The solution is to export the Layer as an ESRI Shapefile.
We may need to export also the SLD defined for the Layer. We must store back also the Layer Styles on the storage.
Once we got all the source data, the final step is to save the Layer Metadata. All the information defined for that Layer must be replicated back on the target instance.

The exercise we are going to execute in 4 steps, consists in:

Download a Layer as ESRI Shapefile: We will download the Layer san_andres_y_providencia_coastline, already configured in GeoNode, as an ESRI ShapeFile. This allows us to store the data in a portable format.
Save and exrpot the Layer SLDs: We will check what are the Styles associated to the san_andres_y_providencia_coastline Layer, retrieve the corresponding SLD files on the GeoServer Data Dir and save them for a later import.
Save and exrpot the Layer Metadata: We will save the san_andres_y_providencia_coastline Layer metadata as an ISOTC211/19115 XML file for a later import.
Import back the Layer: We will cleanup the GeoNode instance, being sure that also GeoServer will be cleaned, and then we will restore back the san_andres_y_providencia_coastline Layer using the fiels saved during the previous steps.

1. Download a Layer as ESRI Shapefile¶

Note

In order to execute this exercise, we assume that you have read all the previous sections and your source GeoNode has a Layer already configured as specified on the preliminary notes.

Excercise

Export A Layer As A SHAPEFILE

Warning

You must be GeoNode Administrator in order to successfully execute this exercise.

Identify the real Layer name.

Connect to your local GeoNode instance and explore the available Layers.

GeoNode Explore Layers

Select the Layer San Andreas Coastlines and click on Metadata Edit.

GeoNode Edit Metadata

You will find the real Layer name as subtitle of the Edit Metadada page. Note it somewhere.

GeoNode Layer Real Name
Identify the GeoServer Endpoint.

Identify the GeoServer endpoint. Click on admin and the on GeoServer menù link.

GeoServer Endpoint

In our case the GeoServer Endpoint is http://localhost/geoserver.

GeoServer Web Page

Export the Layer as an ESRI ShapeFile file format.

Open a Terminal window and go to /home/geo/Desktop/.

$> cd /home/geo/Desktop/

Terminal: Desktop Folder

Create a new folder backup and enter it.

$> mkdir backup
$> cd backup

Terminal: Backup Folder

Finally export the geonode:san_andres_y_providencia_coastline as ShapeFile from GeoServer using the WGET utility command.

$> wget --user=admin --password=admin -O san_andres_y_providencia_coastline.zip "http://localhost/geoserver/geonode/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=geonode:san_andres_y_providencia_coastline&outputFormat=SHAPE-ZIP"

Terminal: Save san_andres_y_providencia_coastline.zip

Note

Lets examine the command we just executed:

# This is the executable
wget

# Those are the GeoNode credentials for the ``admin`` user. Yours maybe different.
--user=admin --password=admin

# This is the name of the output file we want to create. It's not important, you can choose anyone.
-O san_andres_y_providencia_coastline.zip

# This is the full URL to let GeoServer save the source data as ShapeFile ZIP.
"http://localhost/geoserver/geonode/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=geonode:san_andres_y_providencia_coastline&outputFormat=SHAPE-ZIP"

# 1. It must be quoted ""
# 2. The first part is the GeoServer Endpoint "http://localhost/geoserver"
# 3. The "typename" is the real GeoNode Layer name "typeName=geonode:san_andres_y_providencia_coastline"
# 4. With "outputFormat" we say to GeoServer how to download data "outputFormat=SHAPE-ZIP"

Unzip the file into the backup folder.

$> unzip san_andres_y_providencia_coastline.zip

Terminal: Unzip san_andres_y_providencia_coastline.zip

Check that the ShapeFile has been correctly downloaded. You must see the 4 .dbf, .prj, .shp, .shx files on the backup folder.

$> ls -la

Terminal: Check san_andres_y_providencia_coastline.zip

Note

You can also delete the san_andres_y_providencia_coastline.zip file now. It won’t be useful anymore in the future.

2. Save and exrpot the Layer SLDs¶

Note

In order to execute this exercise, we assume that you have read all the previous sections and your source GeoNode has a Layer already configured as specified on the preliminary notes.

Excercise

Export The Layer Default SLD

Warning

By default GeoNode creates an SLD on GeoServer with the same name of the imported Layer and use it as default style. In this exercise we will assume that the Layer uses only the default GeoNode SLD.

Identify the full path of the GeoServer Data Dir

Click on the GeoServer link of the GeoNode admin menu.

GeoServer Admin Page

Click on the Server Status link of the GeoServer admin page.

GeoServer Admin Page Server Status

Copy the full path of the GeoServer Data Dir from the Server Status page.

GeoServer Admin Page GeoServer Data Dir

From the Terminal window, copy the san_andres_y_providencia_coastline.sld into the backup folder.

$> cd /home/geo/Desktop/backup/
$> cp /var/lib/tomcat7/webapps/geoserver/data/styles/san_andres_y_providencia_coastline.sld .

Terminal: Copy san_andres_y_providencia_coastline.sld

Note

Lets examine the command we just executed:

# This is the executable
cp

# This is the full path of the ``san_andres_y_providencia_coastline.sld`` file
/var/lib/tomcat7/webapps/geoserver/data/styles/san_andres_y_providencia_coastline.sld

# 1. Notice that the first part ``/var/lib/tomcat7/webapps/geoserver/data`` is the path of the GeoServer Data Dir
# 2. You must add the path ``/styles/`` to the path of the GeoServer Data Dir
# 3. The SLD file has the same prefix of the original Layer ``san_andres_y_providencia_coastline.sld``

Check that the SLD has been correctly downloaded. You must see the .sld file on the backup folder.

$> ls -la

Terminal: Check san_andres_y_providencia_coastline.sld

3. Save and exrpot the Layer Metadata¶

Note

In order to execute this exercise, we assume that you have read all the previous sections and your source GeoNode has a Layer already configured as specified on the preliminary notes.

Excercise

Export A Layer Metadata As An ISOTC211/19115 XML

Warning

You must be GeoNode Administrator in order to successfully execute this exercise.

Get the ISOTC211/19115 XML URL.

Connect to your local GeoNode instance and explore the available Layers.

GeoNode Explore Layers

Select the Layer San Andreas Coastlines and click on Download Metadata.

GeoNode Download Metadata

Click with the right mouse button over the ISO link and then on Copy Link Location.

Copy Link Location Of ISOTC211/19115 XML

Store the ISOTC211/19115 XML through the WGET command.

From the Terminal window, goto the backup folder of the san_andres_y_providencia_coastline file we created before.

$> cd /home/geo/Desktop/backup/

Finally export the san_andres_y_providencia_coastline.xml as ISOTC211/19115 XML from GeoNode using the WGET utility command.

$> wget --user=admin --password=admin -O san_andres_y_providencia_coastline.xml "http://localhost/catalogue/csw?outputschema=http%3A%2F%2Fwww.isotc211.org%2F2005%2Fgmd&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=3236a2e0-f023-11e5-86e3-08002779b53d"

Terminal: Save san_andres_y_providencia_coastline.xml

Note

Lets examine the command we just executed:

# This is the executable
wget

# Those are the GeoNode credentials for the ``admin`` user. Yours maybe different.
--user=admin --password=admin

# This is the name of the output file we want to create. **It's important** that the file has the same prefix of the ``san_andres_y_providencia_coastline`` Layer name.
-O san_andres_y_providencia_coastline.xml

# This is the full URL of the ISOTC211/19115 XML.
"http://localhost/catalogue/csw?outputschema=http%3A%2F%2Fwww.isotc211.org%2F2005%2Fgmd&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=3236a2e0-f023-11e5-86e3-08002779b53d"

# 1. It must be quoted ""
# 2. The first part is the GeoNode Catalogue Endpoint "http://localhost/catalogue"
# 3. With "outputschema" we say to GeoNode how to download metadata "outputschema=http%3A%2F%2Fwww.isotc211.org%2F2005%2Fgmd"
# 4. Notice that any Layer in GeoNode is identified by a unique ID "id=3236a2e0-f023-11e5-86e3-08002779b53d", which can be retrieved from the Layer Metadata panel.

Check that the XML has been correctly downloaded. You must see the .xml file on the backup folder.

$> ls -la

Terminal: Check san_andres_y_providencia_coastline.xml

4. Import back the Layer through the “importlayers” GeoNode Management Command¶

At this point we have everything we need to restore the Layer saved into the /home/geo/Desktop/backup/ folder.

What we are going to do now is:

Cleanup GeoNode and GeoServer, being sure that the san_andres_y_providencia_coastline has been completely removed from all instances.
Restore the san_andres_y_providencia_coastline and its Metadata
Restore the san_andres_y_providencia_coastline SLD Style

Exercise

Cleanup The san_andres_y_providencia_coastline Layer

Delete The san_andres_y_providencia_coastline Layer.

Connect to your local GeoNode instance and explore the available Layers.

GeoNode Explore Layers

Select the Layer San Andreas Coastlines and click on Layer Remove.

GeoNode Layer Remove

Confirm that you want to remove the san_andres_y_providencia_coastline Layer.

GeoNode Layer Remove Confirm
Double check that the Layer has been removed from GeoServer also.

Connect to your local GeoServer instance Admin GUI, from GeoNode admin menu.

GeoServer Admin GUI

Double check that the Layer san_andres_y_providencia_coastline is no more present on GeoServer Layer list. If so delete it manually.

GeoServer Admin GUI: Layers

Double check that the Style san_andres_y_providencia_coastline is no more present on GeoServer Style list. If so delete it manually.

GeoServer Admin GUI: Styles
Once GeoNode and GeoServer are cleared, import back the san_andres_y_providencia_coastline Layer through the “importlayers” GeoNode Management Command

From the Terminal window, goto the backup folder of the san_andres_y_providencia_coastline file we created before.
```
$> cd /home/geonode/geonode/
```
Finally import the san_andres_y_providencia_coastline Layer using the “importlayers” GeoNode Management Command.
```
$> python manage.py importlayers /home/geo/Desktop/backup/
```
Terminal: Import san_andres_y_providencia_coastline
Note

Notice that
- The “importlayers” GeoNode Management Command automatically imports all the ShapeFiles available into the backup folder
- The “importlayers” GeoNode Management Command automatically restores the Layer metadata if the ISOTC211/19115 XML is available on the same folder and has the same layer previx
- The “importlayers” GeoNode Management Command automatically restores the Layer styles if the SLD is available on the same folder and has the same layer previx
Double check that the san_andres_y_providencia_coastline Layer has been correctly restored into GeoNode.