You need to open the file with the intent to both read and writing to it, if this operation fails then it means the file is in use.
Code:
function IsFileInUse(Filename: String): Boolean;
begin
Result := False;
try
with TFileStream.Create(Filename,fmOpenReadWrite) do // this should fail
try
finally
Free;
end;
except
Result := True; // this handles the exception and tells you the file is in use
end;
end;