But one thing happened there which might have struck fear into them. For when a woman out of Cook's Street had come to take the price of the meal she had sold them she looked into the cellar close to them, and saw a most horrible devil, as though he were writing. And all of a tremble at his horrible countenance, she drew back the foot which she had put across the threshold, but not without punishment, for instantly, that foot was roasted by infernal fire, and fell rotting from her body'.

In the late 15th and 16th centuries, there are several reference to Moyse's Hall being owned by a family called King. For example, in 1514 a merchant called Richard King bequeathed to his wife Cecily my place and tenement that I dwell in called Moyse's Hall.

By 1619 Moyse's Hall was owned by a gentleman called Henry Collinge and in 1626 it was sold to the Guildhall Feoffment Trust. This Trust was a charity, originally established by the medieval Candlemas Guild, which administered a number of charitable bequests made to the town. After the dissolution of the abbey, the Trust assumed an important role in the governing and maintenance of the town. The Feoffees used Moyse's Hall as the town gaol, the workhouse and the bridewell, or house of correction. It housed both poor people and offenders.

The Feoffees kept good records of their properties and a number of documents survive from the 17th and 18th centuries describing the items bought for the inhabitants, the people appointed as keepers and the repairs made to the building. Shoes and stockings were bought for the inmates and woollen cloth for their clothes.

These included 'shyftes', 'a petycote', 'a wastecote', 'a dublette and hose', 'a shirte', 'a curtll and wastecote', 'aprons', 'smocks, and 'ecke clothes'. Rye straw was bought for their beds and linen cloth for sheets. Tools were bought for their work of preparing hemp. Some food was bought, such as bread and beer, but not a great deal: one account notes sadly 'iis to buy breade, they had noe worke tht day beinge Saterdaye nor nothinge to eat upone Sundaye'

By the 1730's, the number of inhabitants was apparently too great for one building, and the workhouse was moved elsewhere. After this Moyse's Hall was generally referred to as the Bridwell. There was another prison on the west side of Cornhill which had been used by the Abbot before the Dissolution. It continued to be used as a prison until 1804 when a new gaol on Sicklesmere Road was opened. It appears that serious offenders were housed in the prison, whilst more minor offenders (vagrants, drunkards, prostitutes, etc) were sent to Moyse's Hall. Clearly, however, Moyse's Hall was not as well-used as formerly, for the Guildhall Feoffees sold off some parts of the building in 1812. In 1836 a police force was established in Bury St Edmunds, and Moyse's Hall became the police station. Even then, part of it was still used to house offenders.

In 1791 a proposal was made to erect a turret clock on the top of Moyse's Hall. It was a handsome structure with a cupola and is shown in some early prints. In 1858, considerable restoration work was carried out to designs by George Gilbert Scott, the then fashionable Victorian architect who designed the Albert Memorial in London. Many features of the building can be dated to this restoration: in particular, the clock turret with its cupola was taken down and replaced by the turret that survives today.

In 1892, the police station moved from Moyse's Hall to new headquarters in St John's Street, and the idea of converting Moyse's Hall into a museum was first mooted. It was not immediately accepted, however. In the intervening period, part of the building was used by the Great Eastern Railway Company as a parcels and enquiry office. In 1894 the Guildhall Feoffees transferred both Moyse's Hall and the Guildhall to the Borough Council. The Borough Council proposed to turn Moyse's Hall into a fire station. Although this proposal was greeted by public protest, there is some evidence that it had already been used for storing fire-fighting equipment.

An old photograph shows a row of hoses and other equipment probably used for fire-fighting, and a record of 1779 states that the Guildhall Feoffees gave permission 'for the making of a proper place for keeping one of the fire engines in some commodious part of the premises under the Bridewell'. The fire station proposal was eventually abandoned and the museum idea gathered strength.

In May 1899, the museum was officially opened by Lord John Hervey. The collections displayed came from three main sources. The Bury St Edmunds and West Suffolk Archaeological Institute had collected together many artefacts and had presented these to the Borough Council in 1878, since when they had been stored out of sight at the Athenaeum. The master of the workhouse in College Street (successor to the workhouse in Moyse's Hall) had collected Bury St Edmunds books, keys and coins, and these were bought. The first curator, J Jennings, arranged a collection of furniture, weapons, clothing, ornaments, etc., all lent by local people. Early pictures of the museum interior show these borrowed articles, some of which look very fine.

The earliest surviving detailed map of Moyse's Hall, Thomas Warren's map of 1741, shows that the Guildhall Feoffees also owned the property next to Moyse's Hall (to the left of the entrance front) and that this was a public house called the Castle Inn. Indeed, part of Moyse's Hall itself, the rooms to the left of the modern entrance, was used by the inn. Warren calls it 'The Mansion House belonging to the Castle'. In 1812, the Guildhall Feoffees sold the whole of the Castle Inn, including the Moyse's Hall rooms. However, when Moyse's Hall was converted into a museum in 1899, this part was bought by the Borough Council from Greene King Brewery and the Norman rooms were reunited as a single building. The Castle Inn has now been gutted, but a beer cellar still survives under Moyse's Hall as a reminder of its existence.

At the rear of the Norman core of Moyse's Hall is an L-shaped range of buildings now known as 41 Cornhill, part of which dates back to the early 16th century. There is some architectural and historical evidence to suggest that it was attached to Moyse's Hall from its earliest stages.

Thomas Warren's map of 1741 shows that 41 Cornhill was then owned by the Guildhall Feoffment Trust and formed part of the Bridewell. However, the Guildhall Feoffees sold it in 1812, when they also sold the Castle Inn. For much of the 20th century it was used as shops. In 1972 the Borough Council bought it and opened two new museum galleries in the part immediately behind Moyse's Hall. The remaining part of the building has been developed to provide a range of new facilities, including an education room, and a gallery housing the collections of the Suffolk Regiment.