With standard near-surface geothermal energy, boreholes up to 200 meters are drilled. The heat difference from that depth is only around 10-15°C, but that’s enough to collect the energy to heat buildings.

So yes, drilling into lava areas will spice up things.