Michel Krämer

We present GeoR­ocket, a soft­ware for the man­age­ment of very large geo­spa­tial data­sets in the cloud. GeoR­ocket em­ploys a novel way to handle ar­bit­rar­ily large data­sets by split­ting them into chunks that are pro­cessed in­di­vidu­ally. The soft­ware has a mod­ern re­act­ive ar­chi­tec­ture and makes use of ex­ist­ing ser­vices in­clud­ing Elast­ic­search and stor­age back ends such as Mon­goDB or Amazon S3. GeoR­ocket is schema-ag­nostic and sup­ports a wide range of het­ero­gen­eous geo­spa­tial file formats. It is also format-pre­serving and does not al­ter im­por­ted data in any way. The main be­ne­fits of GeoR­ocket are its per­form­ance, scalab­il­ity, and us­ab­il­ity, which make it suit­able for a num­ber of sci­entific and com­mer­cial use cases deal­ing with very high data volumes, com­plex data­sets, and high ve­lo­city (Big Data). GeoR­ocket also provides many op­por­tun­it­ies for fur­ther re­search in the area of geo­spa­tial data man­age­ment.

Smart Cit­ies make use of ICT tech­no­logy to ad­dress the chal­lenges of mod­ern urban man­age­ment. The cloud provides an ef­fi­cient and cost-ef­fect­ive plat­form on which they can man­age, store and pro­cess data, as well as build ap­plic­a­tions per­form­ing com­plex com­pu­ta­tions and ana­lyses. The quickly chan­ging re­quire­ments in a Smart City re­quire flex­ible soft­ware ar­chi­tec­tures that let these ap­plic­a­tions scale in a dis­trib­uted en­vir­on­ment such as the cloud. Smart Cit­ies have to deal with huge amounts of data in­clud­ing sens­it­ive in­form­a­tion about in­fra­struc­ture and cit­izens. In or­der to lever­age the be­ne­fits of the cloud, in par­tic­u­lar in terms of scalab­il­ity and cost-ef­fect­ive­ness, this data should be stored in a pub­lic cloud. However, in such an en­vir­on­ment, sens­it­ive data needs to be en­cryp­ted to pre­vent un­au­thor­ized ac­cess. In this pa­per, we present a soft­ware ar­chi­tec­ture design that can be used as a tem­plate for the im­ple­ment­a­tion of Smart City ap­plic­a­tions. The design is based on the mi­croservice ar­chi­tec­tural style, which provides prop­er­ties that help make Smart City ap­plic­a­tions scal­able and flex­ible. In ad­di­tion, we present a hy­brid ap­proach to se­cur­ing sens­it­ive data in the cloud. Our ar­chi­tec­ture design com­bines a pub­lic cloud with a trus­ted private en­vir­on­ment. To store data in a cost-ef­fect­ive man­ner in the pub­lic cloud, we en­crypt metadata items with CP-ABE (Cipher­text-Policy At­trib­ute-Based En­cryp­tion) and ac­tual Smart City data with sym­met­ric en­cryp­tion. This ap­proach al­lows data to be shared across mul­tiple ad­min­is­tra­tions and makes ef­fi­cient use of cloud re­sources. We show the ap­plic­ab­il­ity of our design by im­ple­ment­ing a web-based ap­plic­a­tion for urban risk man­age­ment. We eval­u­ate our ar­chi­tec­ture based on qual­it­at­ive cri­teria, bench­mark the per­form­ance of our se­cur­ity ap­proach, and dis­cuss it re­gard­ing hon­est-but-curi­ous cloud pro­viders as well as at­tack­ers try­ing to ac­cess user data through eaves­drop­ping. Our find­ings in­dic­ate that the mi­croservice ar­chi­tec­tural style fits the re­quire­ments of scal­able Smart City ap­plic­a­tions while the pro­posed se­cur­ity ap­proach helps pre­vent un­au­thor­ized ac­cess.

We present a dy­namic search­able sym­met­ric en­cryp­tion scheme al­low­ing users to se­curely store geo­spa­tial data in the cloud. Geo­spa­tial data sets of­ten con­tain sens­it­ive in­form­a­tion, for ex­ample, about urban in­fra­struc­tures. Since clouds are usu­ally provided by third parties, these data need to be pro­tec­ted. Our ap­proach al­lows users to en­crypt their data in the cloud and make them search­able at the same time. It does not re­quire an ini­tial­iz­a­tion phase, which en­ables users to dy­nam­ic­ally add new data and re­move ex­ist­ing re­cords. We design mul­tiple pro­to­cols dif­fer­ing in their level of se­cur­ity and per­form­ance, re­spect­ively. All of them sup­port quer­ies con­tain­ing boolean ex­pres­sions, as well as geo­spa­tial quer­ies based on bound­ing boxes, for ex­ample. Our find­ings in­dic­ate that al­though the search in en­cryp­ted data re­quires more runtime than in un­en­cryp­ted data, our ap­proach is still suit­able for real-world ap­plic­a­tions. We fo­cus on geo­spa­tial data stor­age, but our ap­proach can also be ap­plied to ap­plic­a­tions from other areas deal­ing with keyword-based searches in en­cryp­ted data. We con­clude the pa­per with a dis­cus­sion on the be­ne­fits and draw­backs of our ap­proach.

With the grow­ing num­ber of devices that can col­lect spa­ti­otem­poral in­form­a­tion, as well as the im­prov­ing qual­ity of sensors, the geo­spa­tial data volume in­creases con­stantly. Be­fore the raw col­lec­ted data can be used, it has to be pro­cessed. Cur­rently, ex­pert users are still re­ly­ing on desktop-based Geo­graphic In­form­a­tion Sys­tems to per­form pro­cessing work­flows. However, the volume of geo­spa­tial data and the com­plex­ity of pro­cessing al­gorithms ex­ceeds the ca­pa­cit­ies of their work­sta­tions. There is a paradigm shift from desktop solu­tions to­wards the Cloud, which of­fers vir­tu­ally un­lim­ited stor­age space and com­pu­ta­tional power, but de­velopers of pro­cessing al­gorithms of­ten have no back­ground in com­puter sci­ence and hence no ex­pert­ise in Cloud Com­put­ing.

Our re­search hy­po­thesis is that a mi­croservice ar­chi­tec­ture and Do­main-Spe­cific Lan­guages can be used to or­ches­trate ex­ist­ing geo­spa­tial pro­cessing al­gorithms, and to com­pose and ex­ecute geo­spa­tial work­flows in a Cloud en­vir­on­ment for ef­fi­cient ap­plic­a­tion de­vel­op­ment and en­hanced stake­holder ex­per­i­ence. We present a soft­ware ar­chi­tec­ture that con­tains ex­ten­sion points for pro­cessing al­gorithms (or mi­croservices), a work­flow man­age­ment com­pon­ent for dis­trib­uted ser­vice or­ches­tra­tion, and a work­flow ed­itor based on a Do­main-Spe­cific Lan­guage. The main aim is to provide both users and de­velopers with the means to lever­age the pos­sib­il­it­ies of the Cloud, without re­quir­ing them to have a deep know­ledge of dis­trib­uted com­put­ing. In or­der to con­duct our re­search, we fol­low the Design Sci­ence Re­search Meth­od­o­logy. We per­form an ana­lysis of the prob­lem do­main and col­lect re­quire­ments as well as qual­ity at­trib­utes for our ar­chi­tec­ture. To meet our re­search ob­ject­ives, we design the ar­chi­tec­ture and de­velop ap­proaches to work­flow man­age­ment and work­flow mod­el­ling. We demon­strate the util­ity of our solu­tion by ap­ply­ing it to two real-world use cases and eval­u­ate the qual­ity of our ar­chi­tec­ture based on defined scen­arios. Fi­nally, we crit­ic­ally dis­cuss our res­ults.

Our con­tri­bu­tions to the sci­entific com­munity can be clas­si­fied into three pil­lars. We present a scal­able and modi­fi­able mi­croservice ar­chi­tec­ture for geo­spa­tial pro­cessing that sup­ports dis­trib­uted de­vel­op­ment and has a high avail­ab­il­ity. Fur­ther, we present novel ap­proaches to ser­vice in­teg­ra­tion and or­ches­tra­tion in the Cloud as well as rule-based and dy­namic work­flow man­age­ment without a pri­ori design-time know­ledge. For the work­flow mod­el­ling we cre­ate a Do­main-Spe­cific Lan­guage that is based on a novel lan­guage design method.

Cur­rent 3D data cap­tur­ing as im­ple­men­ted on for ex­ample air­borne or mo­bile laser scan­ning sys­tems is able to ef­fi­ciently sample the sur­face of a city by bil­lions of un­se­lect­ive points dur­ing one work­ing day. What is still dif­fi­cult is to ex­tract and visu­al­ize mean­ing­ful in­form­a­tion hid­den in these point clouds with the same ef­fi­ciency. This is where the FP7 IQmu­lus pro­ject enters the scene. IQmu­lus is an in­ter­act­ive fa­cil­ity for pro­cessing and visu­al­iz­ing big spa­tial data. In this study the po­ten­tial of IQmu­lus is demon­strated on a laser mo­bile map­ping point cloud of 1 bil­lion points sampling ~ 10 km of street en­vir­on­ment in Toulouse, France. After the data is up­loaded to the IQmu­lus Ha­doop Dis­trib­uted File Sys­tem, a work­flow is defined by the user con­sist­ing of re­til­ing the data fol­lowed by a PCA driven local di­men­sion­al­ity ana­lysis, which runs ef­fi­ciently on the IQmu­lus cloud fa­cil­ity us­ing a Spark im­ple­ment­a­tion. Points scat­ter­ing in 3 dir­ec­tions are clustered in the tree class, and are sep­ar­ated next into in­di­vidual trees. Five hours of pro­cessing at the 12 node com­put­ing cluster res­ults in the auto­matic iden­ti­fic­a­tion of 4000+ urban trees. Visu­al­iz­a­tion of the res­ults in the IQmu­lus fat cli­ent helps users to ap­pre­ci­ate the res­ults, and de­velopers to identify re­main­ing flaws in the pro­cessing work­flow.