Tankless water heaters, also called combi boilers, instantaneous, continuous flow, inline, flash, on-demand or instant-on water heaters, are also available and gaining in popularity. These water heaters heat the water as the water flows through the device, and do not retain any water internally except for what is in the heat exchanger coil. Point of use tankless (POU) heaters may be installed at the faucet or in the bathroom far from the central water heater, or larger models can be used to provide all the hot water requirements for an entire house.

The high efficiency condensing combi or combination boiler is an increasingly popular choice in UK homes. In fact, combi's now account for over half of all the new domestic boilers installed in Britain.

Tankless water heaters can be broken down into two categories; "full on/full off" and "modulated". Full on/full off units do not change the power output at all. The unit is either on or off. Modulated tankless water heaters base the heat output on the flow of water running through the unit. This is usually done through the use of a 'flow sensor', modulating gas valve, inlet water temperature sensor and an outlet water temperature sensor-choke valve and means that the occupants will receive the same output temperature of water at differing velocities, usually within a close range of ±2 degrees Celsius.

Tankless heaters can be far more efficient than storage water heaters. In both kinds of installation, the absence of a tank saves energy as conventional water heaters have to reheat the water in the tank as it cools off, called standby loss. With a central water heater of any type, water is wasted waiting for water to heat up because of the cold water in the pipes between the faucet and the water heater.

Point of use tankless water heaters are located right where the water is being used, so the water is almost instantly hot, which saves water. They also save even more energy than centrally installed tankless water heaters because no hot water is left in the pipes after the water is shut off. However, point of use tankless water heaters are usually used in combination with a central water heater since they are usually limited to under 6 liters/minute (1.5 US gallons/minute or GPM), as the expense of buying a heater for every kitchen, laundry room, bathroom or sink, often outweighs the money saved in water and energy bills. In addition, point of use water heaters until recently were almost always electrical, and electricity is often substantially more expensive than natural gas.

Under current North American conditions, the most cost effective configuration from an operating viewpoint is usually to use a central tankless water heater for the most of the house, and install a point of use tankless water heater at any distant faucets or bathrooms. However, this may vary according to how much electricity, gas and water costs in the area, the layout of the house, and how much hot water is used. Only electric tankless water heaters were available at first and they are still used for almost all point of use heaters, but natural gas and propane heaters are now common. When consumers are considering a whole house gas tankless unit, they are advised to look at how the unit functions when raising the water temperature by about 42°C (75–77°F). Thus, if they live in a cold weather climate, they are advised to look at the unit's capacity with 3-10°C (38–50°F) inlet water temperatures, and find a size that produces approximately 15 liters/minute (4 GPM) even in winter if they have a typical-sized home and desire what is called a 2-appliance heater. This same unit may produce 25-30 liters/minute (6.3–6.9 GPM) in summer with higher inlet temperatures, but there is greater interest in year round production and usability.

Since the water must be heated instantly, tankless water heaters use a lot of electricity or gas while they are on. If a storage water heater is being replaced with a tankless one, the size of the electrical wiring or gas pipeline may have to be increased to handle the load. Gas units are efficient but require a large volume of gas. In gas appliances both pressure and volume requirements must be met for optimum operation.

Since a tankless water heater is not active when hot water is not being used, they are typically incompatible with hot water re circulation systems or applications where the water is turned off and on repeatedly. There is often a short delay before the water flow detector is triggered and the heating element activated. This can result in a "slug" of cold water when you reopen the hot water tap after having closed it. There are also typically minimum flow requirements that can result in a gap between the coolest warm water that can be achieved with a hot and cold water mix and straight cold water. On the other hand, more sophisticated tankless heaters often come with remote thermostats that can allow the water temperature to be changed interactively to suit the immediate purpose. This is impossible with tank heaters.