MSD: Leveraging CAD Standards

Which came first the Grand Canyon or the river at the bottom of it? Did the Colorado River carve it, or fall into it? By just looking at it certainly seems like the later. I got a chance not only to visit the Grand Canyon on my recent trip, but also followed the headwaters of the Colorado River down the Rockies into Utah until it headed south to the Grand Canyon while I headed west. Southern Utah seems like another even bigger Grand Canyon from the highway…just without a river.

Instead of defining a whole new path of interoperability, the Mapping Specification for DWG, MSD, falls completely within the existing DWG file data constructs and file format to provide a means to let GIS information persist between ArcGIS and AutoCAD workflows. In ArcGIS 9.3 a small change was made to the way CAD files were read to look for a simple query and entity attributes encoded on the entity’s extension dictionary in a DWG file.

I often need to share basemap parcels and attributed infrastructure data from the enterprise GIS with private developers and design contractors that use simple AutoCAD. Using ArcGIS 9.3 these contractors can use the GIS information in plain AutoCAD. Previously the best I could do was create blocks with attributes as points and simple geometry. Sharing attributed lines and polygons was always awkward in plain AutoCAD.

In AutoCAD I can add information exported from ArcGIS. The GIS feature classes when exported create MSD feature class definitions and a copy of the GIS schema in AutoCAD for me to use. I am now able to populate attributes on standard CAD entities using that schema. Inside AutoCAD I can simply add to the data on the AutoCAD layer that is used as the MSD feature class definition criteria. If ArcGIS ever opens the DWG file again it will understand the new information as belonging to that MSD CAD feature class.

Feature Class groupings in MSD are based on a simple CAD property-based filtering query. It is very similar to the concept of a definition query on a CAD feature class added to ArcMap. Instead of creating data in a specific data or custom object somewhere in CAD, I define which entities in a CAD file participate in the feature class. Entities that satisfy the filtering criteria are considered part of the feature class.

I think it is best to use the most basic criteria to define the MSD groupings such as CAD layer names, however more complex criteria can be used. Since most CAD Standards use Layer names as the primary method of organizing data now, by defining an MSD feature class based on CAD layers or combinations of CAD Layers, I can build GIS-ready data with only minor changes to my existing CAD workflows. MSD feature classes are usable in an AutoCAD drawing either by adding the MSD feature class definition within AutoCAD, by adding entities to data exported from ArcGIS, or by utilizing a seed file with the feature class definitions already defined from either or both sources.

What's New in ArcGIS 9.3 for CAD: MSD

I just got back from my first visit to the Grand Canyon. I have flown over the Grand Canyon countless times, but this time I walked right up to the edge of it. I was surprised at the many spots in the park’s public viewing areas that you can literally walk right up to the edge of the Grand Canyon without guard rails and look down! In some ways it was refreshing to see that people were trusted with some level of personal responsibility for not falling in, but at the same time as a parent of an active 4 yr old it was a little disconcerting!

I will try to resist metaphors for closing the chasm between GIS and CAD and simply introduce this series on what is new in ArcGIS 9.3 desktop for CAD. The list of new features in ArcGIS 9.3 desktop for CAD are few in number, however the magnitude, and scope of new uses for these features are many. I will expand on some of these in a series of posts following this one.

I will also make some references to the next release of, ArcGIS for AutoCAD here, which will be ready a short time after the general release of ArcGIS 9.3 desktop. ArcGIS 9.3 desktop is scheduled to start shipping at the beginning of July 2008.

Improvements in ArcGIS 9.3 desktop for CAD address the two most common needs expressed by GIS users for AutoCAD; the first, how do I share my GIS data with AutoCAD users? And secondly, how should I organize data in AutoCAD to make it the most useful in ArcGIS?

ArcGIS 9.3 now reads and writes GIS feature classes from and to AutoCAD files using a technique that employs standard AutoCAD extensible data constructs. ESRI’s implementation of this DWG encoding technique is called the Mapping Specification for DWG or MSD. If I create data in a DWG file using this data encoding technique I can specify GIS feature classes in AutoCAD files using standard AutoCAD drawing entities and data structures. The drawings are still plain DWG data with CAD data objects to define how the entities should be grouped and entity-specific feature attributes.

MSD’s main features are:

• The ability to store a named coordinate system inside the AutoCAD file
• The ability to define which entities in a drawing qualify for a specific GIS feature class
• The ability to define a set of attributes for a given GIS feature class
• The ability to store feature attributes on entities in the drawing

The ArcGIS 9.3 EXPORT TO CAD geoprocessing tool creates MSD-defined GIS feature classes whenever GIS feature classes are written to AutoCAD files. Instead of forcing my AutoCAD friends to understand a Shapefile, other intermediate GIS formats, or buy ad-on products, I now give them attributed GIS feature classes directly usable in their native AutoCAD environment.

Future versions of the no-cost ArcGIS for AutoCAD will include ESRI’s version of tools to create, edit and view MSD stored feature classes within the DWG file. I can also build tools with my own AutoCAD API’s to work with the MSD data. The new ESRI Resource Center ArcGIS 9.3 content will soon include a detailed specification and examples of the DWG data constructs used for MSD.

ArcGIS 9.3 desktop, ArcMap/ArcCatalog will recognize MSD feature classes whenever it reads an AutoCAD file (dwg/dxf) with data is encoded according MSD. Working with the MSD-defined CAD feature classes will be exactly the same as working with the standard POINT, POLYLINE, POLYGON, MULTIPATCH and ANNOTATION CAD feature classes. MSD-defined CAD feature classes will appear along with these standard CAD feature classes and will include just those attributes I have defined for the MSD feature class.

Because MSD leverages the existing CAD techniques I already use in my CAD standards to organize data by layers, color and other symbolic variations, I don’t have to require a change in the way my draftsman work. The MSD feature class definition is like a stored query, or the definition query of an ArcMap layer. MSD feature classes are simply pre-filtered CAD data with their own feature attributes stored within a DWG file.

More to come…

Donuts, Whole and Holes

My favorite donut is a french crueler. I like the thought of apple fritters (often the size of the average US county). I am seldom ever glad afterwards that I ate one. I like chocolate covered cream filled donuts. For whatever reason, I am convinced I could easily eat an entire dozen Krispy Kreme donuts and still be lifting the lid of a closed box to see if there are any more. Maple bars are pretty good too. I’m not sure I understand what the hard cake donut or sprinkles have to do with anything… no accounting for taste I guess. I think the purpose of the sprinkle-donut’s existence, is the same as the black licorice gumball, it exists to mock anyone who is too late to choose something else.

GIS Polygons come in various forms. The solid polygon, like a maple bar or apple fritter, the polygon/donut with one or more holes, a polygon surrounding another polygon (filled donut), and a collection of polygons considered one polygon (One Krispy Kreme just doesn’t seem like enough). GIS Polygons can also be overlapping and non-contiguous.

In CAD, SOLIDS and HATCHES are the real polygons; closed POLYLINES are really linear boundaries, not the polygon itself, but still interpreted as polygons by ArcGIS. It is more common in CAD not to worry about closed polylines at all, and just draw the linear boundary network that can be visually inferred as polygons.

To create donuts and multipart polygons (like the Hawaiian Islands as one feature) in CAD, which are in turn interpreted as such in GIS I can create an AutoCAD block of the multiple entities like closed POLYLINE, CIRCLES, etc... ArcGIS consider all of the sub-geometries of CAD blocks to be a single feature with multiple parts. Multiple points, lines or polygon geometries can still be a single entity in a CAD block insert, which are interpreted as mulit-part features by ArcGIS. The CAD BLOCK INSERT is also considered a single point whose geometry is the insertion point of the BLOCK INSERT entity. I typically use blocks as symbols rather than as primary geometry, but this is a useful exception. Both are valid uses of blocks in CAD.

A way to generate donuts from standard concentric CAD polygons is to use the ArcGIS FEATURE TO POLYGON tool. It will cut-out the holes making both the donut and the hole a new polygon. This is the same tool that can create polygons from a set of lines. The tool will break multiple polygons into single polygons, so a combination of using COPY FEATURES and FEATURE TO POLYGON might be required to get my desired result.

GIS and CAD Signs, Symbols and Fonts

While driving to Oklahoma for the National Christian Homeschool basketball championships this month, I and three other parents rotated driving duties for the 21hr non-stop trip along much of Route 66 (I-40) from California. I pulled shifts in the overnight hours getting to watch the sunrise over the steering wheel, and one shift included a morning snow storm near Flagstaff on the way home. My daughter snapped the accompanying picture in the middle of the night at one of our driver rotation stops somewhere in No-Where New Mexico. Looks like the sign has seen a hard life and was even reused from some signage in days gone by.

At the tournament our girls were one of the youngest teams in the 18yr old and under division and won 3 of their 5 tournament games to finish 21st. Finishing up one spot higher than their initial ranking with a loss to the Indiana state champions and a good team from Missouri that fielded a talented All-American player. We met coaches, parents and young people from all around the country. We had a nice time talking with one team from Wisconsin earlier in the week that we ended playing in our last game. We even sat with them at the awards ceremony. We asked the Wisconsin girls if they had enjoyed meeting other girls from around the country, their point guard gave us a troubled and confused look and responded, 'Girls!?... we’re meeting BOYS!'

One of the primary point symbol types supported by ArcMap is the use of true type-fonts as marker symbols. I use true-type marker symbols in both ArcGIS and CAD when I want the symbols to be the same in both the CAD and GIS versions of converted data. Although I have more flexibility in the creation of cells and blocks for CAD symbology, the ability to reference existing ArcGIS marker symbols in a true-type font can be easier than recreating a CAD symbol set of the GIS marker symbols for interoperability. The trick to getting the ESRI marker symbols in CAD is to create CAD text rather than points. The ArcGIS Export to CAD tool available with an ArcInfo license, allows me to export point features as CAD text by overriding the default [CADType] with TEXT and then specifying the Unicode character or formatting code string in the text value stored in the [TxtValue] field of the point feature layer’s table.

I can specify the right symbol with CAD formatting syntax in the text string recognized by CAD (ie: [\U+0073], or use the keyboard short cut to type in the actual Unicode value in the string once I’ve done a little research to find the values of the symbols I want to use. Finally to get the right symbol in CAD I need to have access to the same true type font in ArcGIS as in the CAD system. Export to CAD will then use the true-type font referenced by the [TxtFont] field that I include in the feature layer attribute table or an existing CAD font style based on that true-type font that I could reference in a field called [TxtStyle].

Luckily we don’t have to travel for the final games of the year. Wish us luck as we play this week for the California State Tournament Championship.

Import From CAD and Chainsaws

When someone is talking on their cell phone in the car and you’re riding along, you really can’t help but eavesdrop on the conversation, albeit one-sided. I recently heard this from a friend of mine while talking with his 8yr old son on the phone:

…Did you get all the leaves picked up? ...How about sweeping the back porch? ...and the grass?.... the what?, …why? ... No buddy, you don’t need a chainsaw for that!

The girls in the back seat immediately tuned in at that last exchange. His sister quickly asked, “Chainsaw?! What is Riley doing with a CHAINSAW?!”

The right tool for the job may be less powerful and simpler than the one I might first think. There is a geoprocessing tool in ArcGIS called Import from CAD. Judging by its name, many people gravitate to the tool to make use of CAD files in ArcGIS. Its original design was to generate a normalized view of a CAD file for dissecting and reassembling into all manner of difficult and sophisticated workflows. However for most purposes it is more than overkill, it is much more work than simply opening and using a CAD file directly.

ArcGIS reads CAD files as GIS content and the process of Importing is not necessary. I can create GIS data sets like Shapefiles, or Geodatabases simply by opening a CAD file and copy-and-pasting or, using any number of tools that copy data such as Copy Features, Merge, Append, Feature Class to Feature Class, etc…

The Import From CAD tool creates a Staging Geodatabase that is a highly normalized set of tables that mimics the actual internal organization of a CAD file. The imeadiate result is less directly usable than the default CAD feature classes created by ArcGIS when it opens a CAD file. In fact when I do use the Import From CAD tool the first thing I usually look to do is link the tables together in a useful way, then create a subset of the data, and copy that data to a Geodatabase. Opening up a CAD file and using the Feature Class to Feature Class tool does that for me in a single step.

Import from CAD is a tool that does handles SPLINE geometry better than the default CAD file direct read, and can access AutoCAD Extended Entity Data. However, if I am not working with this type of data there is little reason for me to bother with the tool.

In like manner, just because a chainsaw can make short work of a rose bush, it may be just a little overkill, resulting in even more pieces for me to pick up after I’m done.

GIS and CAD Windfall

The fortune cookie after my dinner the other night claimed that I would enjoy a windfall. There was some discussion around the table as to what the definition of a windfall was, and the origins of the term.

Webster’s defines it thus:
1 : something (as a tree or fruit) blown
down by the wind

2 : an unexpected, unearned, or sudden
gain or advantage.

I didn’t think much of it at the time, but when I was taking the trash cans out to the curb that windy night and a lotto ticket came cart-wheeling along the ground to my feet,... I was intrigued. It was trash night so scraps of paper blowing around wouldn’t necessarily be uncommon, but a lotto ticket!? Surely the stars were aligning and good things were happening!

I wasn’t sure if the lotto drawing had already taken place or if the ticket was still alive. The data on the ticket in itself was not enough to tell me much of anything. However a quick check on the internet to the lotto website winining-number-server gave me my answer.

Two independent pieces of data that just happen to show up in the same place, although intriguing don’t necessarily mean anything at all. GIS and CAD interoperability is more easily trusted when I get my information right from the source, whether that source is GIS or CAD. When in ArcGIS I can use the contents of a CAD file without creating a copy of it. I certainly can make a copy of it in a GIS format if needed, but sometimes CAD file is the proper source and should be included in a map as such. The same can be true when working in CAD, getting a copy of data from the GIS basemap in a CAD format although useful can be prone to getting out of date or losing the context of how it is used in a map. How GIS data is symbolized is a large component of the value of GIS-generated maps. Directly accessing the GIS basemap from ArcGIS Server using ArcGIS for AutoCAD is one way to ensure I've got the most up to date and relevant information in its proper display format inside AutoCAD.

There is a movie of ArcGIS for AutoCAD on the same page where I can download the application for free. For more information on working with CAD files inside ArcGIS you might consider some of my past articles on the subject.

Alas the lotto ticket was no winner… perhaps I should have played the numbers printed on the back of the fortune cookie fortune instead… hmmm... I’ll let you know!

Georeferencing and the Road to Oklahoma

Basketball season is in full swing with multiple games most weekends. This year we’re fielding two teams a JV and a Varsity team. Like last year our varsity team still includes all underclassmen one 11th grader, three 9th graders and an 8th grader. In the first round of league games the girls remain undefeated. This year the girls are attending both the California state tournament and the National tournament in Oklahoma as an 18 and under team (last year they won the California tournament as a 14 and under team).

Several different divisions will be hosted at the National homeschool event. Our girls will be competing in the 18 and under all-comers group against various teams from around the country. There is a higher division we will not play, seeded from regional tournaments which field teams more at the AAU club level of play. Regardless we will be stretching out from our local athletics organization to a bigger collective enterprise. We will likely participate in contests against teams that out match us in age, skill, speed, strength and experience. This is nothing new for the girls; it has been true for these girls all the years I have coached so far, albeit now they will set their sights on a higher level of play. There they will be integrated into the bigger picture: some of the athletes competing in Oklahoma will go on to play basketball at major universities.

Often a CAD file is drawn in a local coordinate system. When the drawing depicts spatial information it can be drawn accurately without having to be tied into a global coordinate system. However there may come a time when that drawing needs to participate at a higher level and be brought together in a mapping context with data from many different sources. Tools in ArcMap can quickly reposition a CAD file and integrate it into the bigger picture without modifying the CAD file.

Before I start I’ll need to know where two points on the CAD file should match up to two points in the map. I use the Georeferencing toolbar in ArcMap. There are several different tools on the toolbar to support different workflows. Here is how I like to use it. First I load the CAD drawing and make sure one of the CAD feature layers are listed in the drop down of the georeferencing toolbar. (Doesn’t matter which CAD layer POINT, POLYLINE, ANNOTATION, POLYGON, etc...) Next I zoom to roughly the place on the map were the CAD drawing is to end up. I don’t bother zooming out to the position of the CAD file, instead I use the fit to display tool on the menu to quickly get the CAD file in the map frame so that I can pick control points. For accurate placement I will turn on my ArcMap snapping so I can pick control points exactly based on existing geometry. If the CAD file needs to be rotated I might use the rotate tool to get the drawing closer to its final position… then only if it makes it easier to pick the points. I sometimes use the interactive scaling tool for the same reason. Most times I just fit to display and pick the four control points and I’m done. I select the update georeferencing option and the tool creates a .WLD file for me that will be read from now on to put my CAD file in the right place.

Now the coordinates are always adjusted into this position. At this time I can choose a coordinate system for my CAD drawing so that if I ever need project its coordinates along with the map or during some geoprocessing operation it will work.