Representing Task and Machine Heterogeneities for Heterogeneous Computing Systems

Shoukat Ali; Howard Jay Siegel; Muthucumaru  Maheswaran; Debra Hensgen; Sahra Ali

doi:10.6180/jase.2000.3.3.09

Representing Task and Machine Heterogeneities for Heterogeneous Computing Systems

Shoukat Ali¹ , Howard Jay Siegel¹ , Muthucumaru Maheswaran This email address is being protected from spambots. You need JavaScript enabled to view it.² , Debra Hensgen This email address is being protected from spambots. You need JavaScript enabled to view it.³ , and Sahra Ali¹

¹School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285 USA
²Department of Computer Science, University of Manitoba, Winnipeg, MB R3T 2N2 Canada
³OS Research and Evaluation OpenTV, Mountain View, CA 94043 USA

Received: March 1, 2000
Accepted: September 1, 2000
Publication Date: September 1, 2000

Download Citation: ||https://doi.org/10.6180/jase.2000.3.3.09

ABSTRACT

A distributed heterogeneous computing (HC) system consists of diversely capable machines harnessed together to execute a set of tasks that vary in their computational requirements. Heuristics are needed to map (match and schedule) tasks onto machines in an HC system so as to optimize some figure of merit. An HC system model is needed to simulate different HC environments to allow the study of the relative performance of different mapping heuristics under different circumstances. This paper characterizes a simulated HC environment by using the expected execution times of the tasks that arrive in the system on the different machines present in the system. This information is arranged in an “expected time to compute” (ETC) matrix as a model of the given HC system, where the entry (i, j) is the expected execution time of task i on machine j. The ETC model is used to express the heterogeneity among the runtimes of the tasks to be executed, and among the machines in the HC system. An existing range-based technique to express heterogeneity in ETC matrices is described. A coefficient-of-variation based technique to express heterogeneity in ETC matrices is proposed, and compared with the range-based technique. The coefficient-of-variation-based ETC generation method provides a greater control over the spread of values (i.e., heterogeneity) in any given row or column of the ETC matrix than the range-based method.

Keywords: distributed computing, heterogeneous computing, workload characterization, modeling computer systems heterogeneity, modeling workload heterogeneity, cluster computing, grid computing

REFERENCES

[1] Armstrong, R., “Investigation of Effect of Different Run-Time Distributions on Smart-Net Performance,” Master’s thesis, Naval Postgraduate School (1997) (D. Hensgen, Advisor).
[2] Braun, T. D., Siegel, H. J., Beck, N., Bölöni, L., Maheswaran, M., Reuther, A. I., Robertson, J. P., Theys, M. D., Yao, B., Freund, R. F., and Hensgen, D., “A Comparison Study of Static Mapping Heuristics for a Class of Meta-Tasks on Heterogeneous Computing Systems,” 8th IEEE Heterogeneous Computing Workshop (HCW ’99), pp. 15–29 (1999).
[3] Dietz, H. G., Cohen, W. E., and Grant, B. K., “Would You Run It Here... Or There? (AHS: Automatic Heterogeneous Supercomputing),” 1993 International Conference on Parallel Processing (ICPP ’93), Vol. II, pp. 217–221 (1993).
[4] Eshaghian, M. M., ed., Heterogeneous Computing, Artech House, Norwood, MA (1996).
[5] Feitelson, D. G. and Rudolph, L., “Metrics and Benchmarking for Parallel Job Scheduling,” in D. G. Feitelson, L. Rudolph, eds., Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, Springer-Verlag, New York, NY, Vol. 1459, pp. 1–15 (1998).
[6] Foster, I., Kesselman, C., eds., The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, San Fransisco, CA (1999).
[7] Freund, R. F., Gherrity, M., Ambrosius, S., Campbell, M., Halderman, M., Hensgen, D., Keith, E., Kidd, T., Kussow, M., Lima, J. D., Mirabile, F., Moore, L., Rust, B., and Siegel, H. J., “Scheduling Resources in Multi-User, Heterogeneous, Computing Environments with SmartNet,” 7th IEEE Heterogeneous Computing Workshop (HCW ’98), pp. 184–199 (1998).
[8] Ghafoor, A. and Yang, J., “Distributed Heterogeneous Supercomputing Management System,” IEEE Computer, Vol. 26, No. 6, pp. 78– 86 (1993).
[9] Hensgen, D. A., Kidd, T., John, D. S., Schnaidt, M. C., Siegel, H. J., Braun, T. D., Maheswaran, M., Ali, S., Kim, J.-K., Irvine, C., Levin, T., Freund, R. F., Kussow, M., Godfrey, M., Duman, A., Carff, P., Kidd, S., Prasanna, V., Bhat, P., and Alhusaini, A., “An Overview of MSHN: The Management System for Heterogeneous Networks,” 8th IEEE Heterogeneous Computing Workshop (HCW ’99), pp. 184–198 (1999).
[10] Iverson, M. A., Özgüner, F., and Follen, G. J., “Statistical Prediction of Task Execution Times Through Analytic Benchmarking for Scheduling in a Heterogeneous Environment,” 8th IEEE Heterogeneous Computing Workshop (HCW ’99), pp. 99–111 (1999).
[11] Jain, R., The Art of Computer Systems Performance Analysis, John Wiley & Sons, Inc., New York, NY (1991).
[12] Kafil, M. and Ahmad, I., “Optimal Task Assignment in Heterogeneous Distributed Com-putting Systems,” IEEE Concurrency, Vol. 6, No. 3, pp. 42–51 (1998).
[13] Lapin, L. L., Probability and Statistics for Modern Engineering, Waveland Press, Inc., Prospect Heights, IL, Second edn. (1998).
[14] Lopez-Benitez, N. and Hyon, J.-Y., “Simulation of Task Graph Systems in Heterogeneous Computing Environments,” 8th IEEE Heterogeneous Computing Workshop (HCW ’99), pp. 112–124 (1999).
[15] Maheswaran, M., Ali, S., Siegel, H. J., Hensgen, D., and Freund, R. F., “Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems,” Journal of Parallel and Distributed Computing, Vol. 59, No. 2, pp. 107–131 (1999).
[16] Maheswaran, M., Braun, T. D., and Siegel, H. J., “Heterogeneous Distributed Computing,” in J. G. Webster, ed., Encyclopedia of Electrical and Electronics Engineering, Vol. 8, John Wiley, New York, NY, pp. 679–690 (1999).
[17] Papoulis, A., Probability, Random Variables, and Stochastic Processes, McGraw-Hill, New York, NY (1984).
[18] Siegel, H. J. and Ali, S., “Techniques for Mapping Tasks to Machines in Heterogeneous Computing Systems,” Journal of Systems Architecture, Vol. 46, No. 8, pp. 627–639 (2000).
[19] Siegel, H. J., Braun, T. D., Dietz, H. G., Kulaczewski, M. B., Maheswaran, M., Pero, P. H., Siegel, J. M., So, J. J. E., Tan, M., Theys, M. D., and Wang, L., “The PASM Project: A Study of Reconfigurable Parallel Computing,” 2nd International Symposium on Parallel Architectures, Algorithms, and Networks (ISPAN’96), pp. 529–536 (Invited paper) (1996).
[20] Singh, H. and Youssef, A., “Mapping and Scheduling Heterogeneous Task Graphs Using Genetic Algorithms,” 5th IEEE Heterogeneous Computing Workshop (HCW ’96), pp. 86–97 (1996).
[21] Topcuoglu, H., Hariri, S., and Wu, M.-Y., “Task Scheduling Algorithms for Heterogeneous Processors,” 8th IEEE Heterogeneous Computing Workshop (HCW ’99), pp. 3–14 (1999).
[22] Yang, J., Ahmad, I. and Ghafoor, A., “Estimation of Execution Times on Heterogeneous Supercomputer Architecture,” 1993 International Conference on Parallel Processing (ICPP ’93), Vol. I, pp. 219–225 (1993).