In the past, the Madre de Dios and Beni valleys were probably one of the migration routes for the Proto-Arawakan, the Proto-Maipuran, and possibly the Proto-Panoan tribes. No archaeological evidence of the Huarayo exists, but according to early chronicles, they were likely vassals of the Inca or perhaps their servants guarding the Anti-soyo, the forested eastern slopes of the Inca Empire. There are indications of intensive contacts between the Huarayo and the Inca via trade or tribute. It was said that the Huarayo were entitled to collect the tribute from other groups for the Inca and also to capture their youths for service in the Inca army. Their knowledge of weaving, raising maize, and the use of the sling is diffused from the Andean Indians. Juan Alvarez Maldonado, the first Spanish conqueror, who descended the Madre de Dios in 1567, used the names "Huarayo" and "Guarayo" in his Relación, but mainly the name "Chuncho."
Huarayo contacts with Westerners began in 1539, when Pedro Anzules de Camporedondo reached the Rio Beni. The expedition of Pedro Candia and Mercedian missionaries Diego de Porres and Diego Martínez came to the region of the upper Inambari from 1587 to 1588. In the seventeenth century missionaries of the Jesuit and Franciscan orders entered the area but were few in number. The reports of missionaries and travelers are confused. The Franciscan mission of La Concepción de Apolobamba was founded in 1690 on the left bank of the Beni; the missions of San José de Uchupiamonas and San Antonio de Ixiamas were established in 1713 and 1721 respectively. Under pressure to assimilate to Western ways, the Huarayo moved nearer to the Río Madre de Dios.
In the beginning of the twentieth century, Dominican missionaries started to operate in the Madre de Dios region. Later, the temporary missions of Lago Valencia (1933), El Pilar (1943), and Fundo Concepción (1950) were founded, and an Adventist mission opened in Palmareal (1972). The second wave of the rubber boom (1941-1945) had a harmful effect upon the Huarayo. After 1955 the remaining Huarayo families from the Tambopata and Inambari regions settled on the left bank of the Madre de Dios. In 1960 they chose a place on the opposite side of the river and built the village of Palmareal. The Huarayo living in the Caserío de Infierno, Chonta, and Riberaita are highly acculturated, whereas the Huarayo in Palmareal are less so.