現在位置首頁 > 博碩士論文 > 詳目
  • 同意授權
論文中文名稱:水情測預報平台與分散式運算 [以論文名稱查詢館藏系統]
論文英文名稱:Integration and performance analysis of flood forecasting and distributed computing systems [以論文名稱查詢館藏系統]
院校名稱:臺北科技大學
學院名稱:工程學院
系所名稱:土木工程系土木與防災碩士班(碩士在職專班)
畢業學年度:103
畢業學期:第二學期
中文姓名:簡英皓
英文姓名:Ying-Hao Jian
研究生學號:101428081
學位類別:碩士
語文別:中文
口試日期:2015/01/16
指導教授中文名:楊元森
指導教授英文名:Yuan-Sen Yang
口試委員中文名:楊元森;張哲豪;連和政
口試委員英文名:Yuan-Sen Yang;Che-Hao Chang
中文關鍵詞:分散式運算
英文關鍵詞:FEWS_TaiwanHTCondorDistributed computing
論文中文摘要:近年來水利署已經可以應用水文氣象觀測整合平台FEWS_Taiwan,達成防汛期間每十五分模擬全台各流域預報目標,但不包含系集預報之運算。FEWS_Taiwan為我國水利署與荷蘭Deltares長年合作發展專用於台灣環境的洪水預報系統。於颱洪期間針對全台河川進行河川水位及流量之預報工作,提供即時預報海象、氣象及水文等資訊給水利署內防救災決策人員。為了解決資料與模式間的各種不確定性與提高預報可靠度,目前FEWS_Taiwan經常採用系集預報的方式而這也是未來的分析趨勢。系集預報帶來大量的運算與資料,也代表著預報時間的延長,降低預報的時效性。雖然FEWS_Taiwan平台本身提供分散式運算機制,但其架構於資料同步時產生多餘的資料傳遞,增加預報時間長度,導致預報時效性降低。
本研究引進美國威斯康辛大學麥迪遜分校研發的HTCondor分散式運算系統,建立FEWS_Taiwan與HTCondor間的介接模式,配合SOBEK二維淹水模式,以2009年莫拉克颱風美濃地區之水文水理分析做為測試案例,分析HTCondor改善預報時間之程度。同時本研究提出以運算單元及系集數量為因素之時間評估公式,用以粗估此整合系統預報之時間長度,作為日後系統硬體規劃與運算性能提升評估之依據。
論文英文摘要:In recent years, Water Resources Agency applies hydrometeorology platform FEWS_Taiwan and have achieved a goal that simulates river basins in Taiwan every 15 minutes in a flood control period if ensemble forecast is not included. FEWS_Taiwan is a hydrometeorology platform which is jointly developed by Water Resources Agency and Netherland’s Deltrares. FEWS_Taiwan offers real-time hydrologic data to the disaster prevention decision maker. To solve the uncertainties between forecast data and models improved forecast reliability, FEWS_Taiwan typically uses ensemble forecast. However, ensemble forecast not only brings much computation and data but increases forecast time, reducing the forecast timeliness. FEWS_Taiwan offers distributed computing architecture, but its architecture runs redundant data transfer when running data synchronization, leading to unnecessary increase of forecast time.
This research employed a distributed computing system, named HTCondor, which is developed by University of Wisconsin-Madison. Furthermore, this work built systematic integration of FEWS_Taiwan and HTCondor, and applied to SOBEK 2D hydraulic models. Through the test case of ensemble forecast of Meinong, Taiwan under Typhoon Morakot, 2009, this study investigated the improvement of integrating HTCondor and FEWS_Taiwan. In addition, this work offers a time evaluation equation to estmate forecast time integrated system for timing investigation as well as possible future hardware enhancement evaluation.
論文目次:摘 要 i
ABSTRACT iii
誌 謝 v
目 錄 vii
表目錄 ix
圖目錄 x
第一章 緒論 1
1.1研究動機與目的 1
1.2文獻回顧 2
1.3研究架構 3
第二章 平台與分散式系統 6
2.1 FEWS_Taiwan水情測預報平台 6
2.2 FEWS_Taiwan Stand Alone 6
2.2.1 FEWS_Taiwan銜接外部模式 9
2.2.2 模式銜接工作流程 10
2.2.3 General Adapter模組 13
2.3 FEWS_Taiwan Live System 14
2.3.1 FEWS_Taiwan Live System各元件間之關係 16
2.4 分散式運算系統:HTCondor 17
2.4.1 HTCondor系統架構 18
2.4.2 HTCondor主要特點 20
第三章 水情測預報平台整合設計 22
3.1 FEWS_Taiwan Workflow 22
3.2 外部模式銜接FEWS_Taiwan所需設定檔 24
3.3 FEWS_Taiwan Live System探討 26
3.3.1 FEWS_Taiwan Stand Alone工作流程 27
3.3.2 FEWS_Taiwan Live System運作模式 29
3.3.3 FEWS_Taiwan Live System系集預報 30
3.4 分散式水情測預報平台FEWS_TaiwanH 32
3.4.1 FEWS_TaiwanH運作模式 35
第四章 FEWS_TaiwanH系統實作 37
4.1 FEWS_Taiwan Live System的建立 37
4.2 HTCondor Pool的建立 40
4.3 FEWS_TaiwanH整合SOBEK案例 42
4.3.1 SOBEK模式整合 42
4.3.2 建立模式於平台執行環境階段 43
4.3.3 平台與模式銜接設定階段 45
4.4 FEWS_TaiwanH 成果探討 53
4.4.1 FEWS_TaiwanH效能比較 56
4.4.2 FEWS_TaiwanH性能評估 61
第五章 結論與建議 65
5.1 結論 65
5.2 後續研究方向與建議 66
參考資料 67
附錄 69
A:FEWS_TaiwanH and FEWS_Taiwan Live System _ 09:00 69
B:FEWS_TaiwanH and FEWS_Taiwan Live System _ 00:00 72
論文參考文獻:[1] Tanenbaum, A. S., & Van Steen, M. 2007. Distributed systems. Prentice-Hall.
[2] Basney, J., Raman, R., & Livny, M. 1999, March. High Throughput Monte Carlo. In PPSC.
[3] Livny, M., Basney, J., Raman, R., & Tannenbaum, T. 1997. Mechanisms for high throughput computing. SPEEDUP journal, 11(1), 36-40.
[4] Carlos A. Varela, Gul Agha, Wei-Jen Wang, Travis Desell, Kaoutar ElMaghraoui, Jason LaPorte,and Abe Stephens. 2007.The SALSAProgramming Language 1.1.2 Release Tutorial.Technical report 07-12,Dept. of Computer Science, R.P.I.
[5] Welcome to Apache Hadoop, https://hadoop.apache.org/
[6] HTCondor, http://research.cs.wisc.edu/htcondor/index.html
[7] Thain, D., Tannenbaum, T., & Livny, M. 2005. Distributed computing in practice: The Condor experience. Concurrency-Practice and Experience, 17(2-4), 323-356.
[8] Frey, J., Tannenbaum, T., Livny, M., Foster, I., & Tuecke, S. 2002. Condor-G: A computation management agent for multi-institutional grids. Cluster Computing, 5(3), 237-246.
[9] 李天浩、陳台琦,2002,水文氣象資訊與水文水理系統模式整合之研發(1/2),台中:經濟部水利署。
[10] 國家高速網路與計算中心,2009,水文氣象觀測整合平台擴充建置(2/2),經濟部水利署。
[11] 陳柏愷,2012,介接淹水模式於即時作業平台,國立台北科技大學土木與防災資訊所碩士論文。
[12] Chappell, D. A., & Monson-Haefel, R. 2000. Java Message Service. " O'Reilly Media, Inc.".
[13] Tannenbaum, T., Wright, D., Miller, K., & Livny, M. 2001, November. Condor: a distributed job scheduler. In Beowulf cluster computing with Linux (pp. 307-350). MIT press.
[14] Wright, D. 2001, June. Cheap cycles from the desktop to the dedicated cluster: combining opportunistic and dedicated scheduling with Condor. In Conference on Linux clusters: the HPC revolution.
[15] Pruyne, J., & Livny, M. 1996, January. Managing checkpoints for parallel programs. In Job Scheduling Strategies for Parallel Processing (pp. 140-154). Springer Berlin Heidelberg.
[16] Tannenbaum, T., Basney, J., & Livny, M. 1997. Checkpoint and migration of UNIX processes in the Condor distributed processing system. Computer Sciences Department, University of Wisconsin.
[17] Coleman, N., Raman, R., Livny, M., & Solomon, M. 2003. Distributed policy management and comprehension with classified advertisements. University of Wisconsin.
論文全文使用權限:同意授權於2015-07-22起公開