Please Come Back Home

Split by Attributes GP tool....when would you actually use this?

There are times when you're making a map but symbolizing using the symbology feature is not enough to characterize the data visually. Thus, having this tool makes cartographical work a little easier by generating copies of the original data, split into separate layers based on the attribute that we need. By doing this, it makes the task of adding the legend much easier in the layout as well.

Most often, when making maps for slide presentation, you would want to segregate data into separate layers with certain uniform values for a certain attribute and a create a new data layer which we can use over and over again.

Although definition query can help with visualizing and showing the features with the attribute value that we want, we may want to create a separate data to avoid compromising the original data or constantly repeating the task of typing/configuring the SQL commands.

This tool is valid for shapefiles and feature classes. Any other data types may need to be converted into those two formats before you can run it. Check out the long-winded demo below:

Since this tool is actually a Python script, it can be integrated into a code for batch geoprocessing or model for iteration over many data layers or interconnection to other tools; automation at its full-on glory! 😁

Malaysia Forest Cover 2020 | WWF-Malaysia

Tool: ArcGIS Pro 2.9.3, Operations Dashboard ArcGIS & ArcGIS Online Technique: Data transformation and geometric calculation

WWF-Malaysia Forest Cover Baseline is a dashboard of forest cover extent status in selected land uses across Malaysia's region, methodology of analysis and resources involved in the exercise.

The WWF-Malaysia Forest Cover Baseline and Forest Cover Key Performance Index (KPI) is a task undertaken by the Conservation Geographical Information System (CGIS) Unit to amass the discrete information of forest cover extent across Malaysia's 3 main region of legislation: Peninsular Malaysia, Sarawak and Sabah. This exercise produces a concise dashboard report in an online platform that describes the processed information on the forest cover status as well as their prospective areas identified for conservation work.

Report can be interactively accessed at the following:

The dashboard can be accessed at Malaysia Forest Cover 2020.

📌 Availability: Retracted in 2021

Community Empowerment Strategy Dashboard 2021 | WWF-Malaysia

Tool: Operations Dashboard ArcGIS, Survey123 for ArcGIS, ArcGIS Online Technique: XLSForm programming, web application development

The northern highland communities of Lun Bawang have been collaborating with WWF-Malaysia under the Sarawak Conservation Programme (SCP) to empower sustainable economies and managing their natural biodiversity through the Community Empowerment Strategy (formerly known as Community Engagement and Education Strategy).

Since 2016, the communities have been actively mapping out their land uses and culturally important locations to delineate their areas of settlement and source of livelihood. Given the close vicinity of their communities to the licensed timber concessions, producing a definitive map is important to preserve and conserve their surrounding natural capitals.

Several outreach has been done and the community mapping effort has been shifted to implement citizen science via the Survey123 for ArcGIS mobile application which is apart of the ArcGIS ecosystem. This enables the local community to collect information despite the lack of network reception and the data can still be synchronized upon availability automatically or manually shared with the field officers.

📌 Availability: Retracted in 2021

Don't break the chain peeps! Reblog cause I'm looking for inspiration for my next masterpiece! 🙇🙇🙇

reblog/like if you’re an active studyblr/langblr

I’ve just unfollowed a bunch of inactive blogs, now that I follow ONLY 54 blogs??? pls reblog/like so I can have an active dashboard and new friends hehehe

📚 Nature in the Heart of Borneo (2020)

Tool: ArcGIS Pro 2.6.1

Technique: Annotation, Labeling and Symbology

A series of maps were created for the book published by WWF-Malaysia and FORMADAT (Forum Masyarakat Adat Dataran Tinggi Borneo) back in 2020 called Nature in the Heart of Borneo.

This book was meant as a guide to some of the natural attractions at Northern parts of Sarawak. If it was clear, Northern Sarawak is where the we have our very own highlanders which consist of primarily the Lundayeh/Lun Bawang, Sa'ban and Kelabit people. Some of the beautiful settlements up in the north that should not be missed are Ba'kelalan and Long Semadoh. They have beautiful homestays and even more beautiful landscapes with trekking activities lined up for tourists. And this is the culmination of ardent passion by my two absolutely wonderful colleagues, Alicia Ng and Cynthia Chin.

Most part of the maps were made using readily available basemap provided by Esri in their Living Atlas. But in entirety, many of the features and details are drawn manually within ArcGIS Pro. Like many other mapmakers out there, the labeling feature is horrendously temperamental and I either end up using annotations instead.

In summary, technically, there are 2 lessons learned here:

1️⃣ Establish concept or pick an idea before you start drawing

A concept of the map and palette should be established at the earliest stage possible. And don't just throw the task of making maps and split them evenly between cartographers. They won't have similar ideas or similar interpretations of the concept. It'll only give you double the pain of creating the maps again from scratch.

2️⃣ Omit borders

If you're making maps for books, don't border trying to make borders and fully utilize the whole layout. In the end, you'll need to export out your maps and they will resize it anyway and it'll compromise the maps you created. As if it wasn't graining enough in the first place, it'll look absolutely microscopic by the time they're done.

home is where you are

mapshaper

Ok. 

I wanna know why have I never heard of this online tool before. Like, what the hell is wrong with the social media? Is something wrong with Twitter or Instagram or something that they never caught on mapshaper? Or was it just me and my hazardous ignorance, yet again?

Have you tried this free nifty online tool that literally simplify crazy complicated shapefile polygons like it’s no one’s business?!

It started with some last minute inspiration on how to collate data from 3 different regions; developed from remote sensing techniques which vary from one another. The common output here is to turn all of them into a vector file; namely shapefile, and start working on the attribute to ease merging of the different shapefile layers.

Once merged, this shapefile is to be published as a hosted feature layer into the ArcGIS Online platform and incorporated into a webmap that serves as a reference data to configure/design a dashboard. What is a dashboard? It's basically an app template in ArcGIS Online that summarizes all the important information in your spatial data. It's a fun app to create, no coding skills required. Check out the gallery here for reference:

Operations Dashboard for ArcGIS Gallery

There are two common ways to publish hosted feature layer into ArcGIS Online platform.

Method 1: Zip up the shapefile and upload it as your content. This will trigger the command inquiring if you would like to publish it as a hosted feature layer. You click 'Yes' and give it a name and et voila! You have successfully publish a hosted feature layer.

Method 2: From an ArcGIS Desktop or ArcGIS Pro, you publish them as feature service (as ArcMap calls them) or web layer (as the its sister ArcGIS Pro calls them). Fill up the details and enabling the function then hit 'Publish' and it will be in the platform should there be no error or conflicting issues.

So, what was the deal with me and mapshaper? 

🛑 A fair warning here and please read these bullet points very carefully:

I need you to remember...I absolve any responsibility of what happens to your data should you misinterpreted the steps I shared. 

Please always  👏🏻  BACK 👏🏻 UP  👏🏻  YOUR 👏🏻 DATA. Don’t even try attempting any tools or procedure that I am sharing without doing so. Please. Cause I am an analyst too and hearing someone else forget to save their data or create a backup is enough to make me die a little inside. 

For this tool, please export out the attribute table of your shapefile because this tool will CHANGE YOUR SHAPEFILE ATTRIBUTES. 

When I was publishing the vector I have cleaned and feature-engineered via ArcGIS Pro...it took so long that I was literally dying inside. I'm not talking about 20 minutes or an hour. It took more than 12 hours and it did not conjure the 'Successfully published' notification as I would've expected from it.

So at around 5.30 am, I randomly type 'simplify shapefily online free'. Lo and behold, there was mapshaper.

All I did was, zip up my polygon, drag it to the homepage and it will bring you to the option of choosing the actions that will be executed while the data is being imported into mapshaper:

detect line intersections

snap vertices

This option will help you to detect the intersections of lines within your vector/shapefile. This can help identify topological error.

The option to snap vertices will snap together points of similar or almost identical coordinate system. But it does not work with TopoJSON formats.

There is something interesting about this options too; you can enter other types of customized options provided by the tool from its command line interface! But hold your horses peeps. I did not explore that because here, we want to fix an issue and we'll focus on that first. I checked both options and import them in.

This will bring the to a page where there you can start configuring options and method to simplify your vector.

To simplify your shapefile, you can have both options to prevent the shape of the polygon being compromised; prevent shape removal, and to utilize the planar Cartesian geometry instead of the usual geoid longitude and latitude; use planar geometry. The implication of the second option is not obvious to me yet since all I wanted was to get the data simplified for easy upload and clean topology, thus, I chose both options to maintain the shape and visibility of all my features despite the highest degree of simplification.

Alike to the options of methodology for simplication in the mainstream software, I can see familiar names:

Douglas-Peuker

Visvalingam / effective area

Visvalingam / weighted area

First and foremost, I had no slightest idea of what these were. Like for real. I used to just go first for the default to understand what sort of output it will bring me. But here, the default; Visvalingam / weighted area, seemed like the best option. What are these methodologies of simplification? There are just algorithms used to help simplify your vectors:

🎯 Douglas-Peucker algorithm decimates a curve composed of line segments to a similar curve with fewer points (Ramer-Douglas-Peucker algorithm, Wikipedia; 2021).

🎯 Visvalingam algorithm is a line simplication operator that works eliminating any less significant points of the line based on effective area concept. That basically means that the triangle formed by each of the line points with two of its immediate neighboring points (Visvalingam Algorithm | aplitop).

🎯 Visvalingam algorithm with weight area is another version of Visvalingam algorithm of subsequent development where an alternative metrics is used and weighted to take into account the shape (Visvalingam & Whelan, 2016).

For reasons I can't even explain, I configured my methodology to utilize the third option and now that I have the time to google it, Thank God I did.

Then, see and play with the magic at the 'Settings' slider where you can adjust and view the simplification made onto the vector! I adjusted it to 5%. The shape retained beautifully. And please bear in mind, this vector was converted from a raster. So, what I really wanted is the simplified version of the cleaned data and to have them uploaded.

Now that you've simplified it, export it into a zipped folder of shapefile and you can use it like any other shapefile after you extracted it.

Remember when I say you have got to export your table of attributes out before you use this tool? Yea...that's the thing. The attribute table will shock you cause it'll be empty. Literally. With only the OBJECTID left. Now, with that attribute table you've backed up, use the 'Join Table' tool in ArcGIS Pro or ArcMap and join the attribute table in without any issues.

Phewh!!

I know that it has alot more functions than this but hey, I'm just getting started. Have you ever done anything more rocket science than I did like 2 days ago, please share it with the rest of us. Cause I gotta say, this thing is cray!! Love it so much.

mapshaper developer, if you're seeing this, I 🤟🏻 you!

UPDATE

I have been asked about the confidentiality of the data. I think this is where you understand the reason behind the fact that they will work even with using just the ‘.shp’ file of the shapefile since _that_ is the vector portion of the shapefile. 

Shapefile is a spatial data format that is actually made up of 4 files; minimum. Each of these files share the same name with different extensions; .prj, .shx, .shp and .dbf. Although I am not familiar with what .shx actually accounts for, the rest of them are pretty straightforward:

.prj: stores the projection information

.dbf: stores the tabulated attributes of each features in the vector file

.shp: stores the shape/vector information of the shapefile. 

So, as the tool indicate, it actually helps with the vector aspect of your data which is crucial in cartography. 

[2022] 30 Day Map Challenge -- FAILED

Last year, I participated once again in the 30 Day Map Challenge that was going around in Twitter-ville come November. It is the 3rd attempt at the marathon and 2022 served as a reminder that progressed too despite getting stuck at Day 3 as life caught up with me.

I don't like the idea that I have left the challenge incomplete, again. It was not my priority and I work better with clear goals or visions of expected output. If it does not add to my need to learn something new ...it will be a task bound to head straight to the backburner. Let's resolve to make it a long-term routine instead of a spurt of stress trying to make the deadline.

As a consequence, I am attuning this task into one that actually gives me the benefit out putting into record the techniques and tools I used to make the maps in writing. I believe that will serve more purpose and added value other than visuals. And perhaps, have some stock ready for submission this year instead.

Anyone else participated in this challenge back in November? How did you do and what would you like to do better for the next one? Don't be shy and do drop a word or two.