Page 1 of 1

Too high record count with QueryRecCount and filter bef open

Posted: Wed 28 Dec 2011 14:20
by cis-wurzen
When querying the record count on a dataset where Options.QueryRecCount is True and a filter was applied to the dataset before it was opened then the record count is too high and does not make sense.

The test case for this is the following:

SQL for the test table

Code: Select all

CREATE TABLE ODACRECORDCOUNTTEST (
  ID   NUMBER(9,0)
);

ALTER TABLE ODACRECORDCOUNTTEST ADD CONSTRAINT ODACRECORDCOUNTTESTPK PRIMARY KEY (ID);
Delphi console application

Code: Select all

program ODACFilteredDSRecordCount;

{$APPTYPE CONSOLE}

uses
  SysUtils,
  DBAccess,
  Ora,
  OraSmart;

{
CREATE TABLE ODACRECORDCOUNTTEST (
  ID   NUMBER(9,0)
);

ALTER TABLE ODACRECORDCOUNTTEST ADD CONSTRAINT ODACRECORDCOUNTTESTPK PRIMARY KEY (ID);
}

const
  VerboseLevel: Integer = 0;//default = 0 (value does not matter for test result)
  UseWhileNotEof: Boolean = False;//default = False (value does not matter for test result)
  OpenBeforeApplyFilter: Boolean = False;//default = False
  QueryRecCount = True;//default = True
  cServer = 'YourServer';
  cUsername = 'YourUser';
  cPassword = 'YourPassword';

type
  TFetchHandler = class(TObject)
  private
    FFetchCount: Integer;
  public
    procedure OraQueryAfterFetch(DataSet: TCustomDADataSet);
    property FetchCount: Integer read FFetchCount;
  end;

function TestQueryRecCountWithFilterAppliedBeforeOpen: Boolean;
var
  se: TOraSession;
  qr: TSmartQuery;
  I, RecordCountAfterOpen: Integer;
  FH: TFetchHandler;
begin
  FH := TFetchHandler.Create;
  se := TOraSession.Create(nil);
  qr := TSmartQuery.Create(nil);
  try
    se.Server := cServer;
    se.Username := cUserName;
    se.Password := cPassword;
    with qr do
    begin
      Options.QueryRecCount := QueryRecCount;
      AfterFetch := FH.OraQueryAfterFetch;
      Session := se;
      SQL.Add('DELETE FROM ODACRECORDCOUNTTEST');
      ExecSQL;
      SQL.Clear;
      SQL.Add('SELECT * FROM ODACRECORDCOUNTTEST ORDER BY ID');
      Keyfields := 'ID';
      Open;
      for I := 1 to 1000 do
      begin
        Insert;
        Fields[0].AsInteger := I;
        Post;
      end;
      Close;
      if OpenBeforeApplyFilter then
        Open;
      Filter := 'ID > 70 AND ID = 1 then
        WriteLn(Format('After Open (RecordCount = %d)', [RecordCount]));
      if UseWhileNotEof then
      begin
        First;
        while not Eof do
        begin
          if VerboseLevel >= 2 then
            WriteLn(Format('Next (RecordCount = %d)', [RecordCount]));
          Next;
        end;
        if VerboseLevel >= 1 then
          WriteLn(Format('After While not Eof (RecordCount = %d)', [RecordCount]));
      end
      else
      begin
        Last;
        if VerboseLevel >= 1 then
          WriteLn(Format('After Last (RecordCount = %d)', [RecordCount]));
        Close;
      end;
    end;
  finally
    qr.Free;
    se.Free;
    FH.Free;
  end;
  Result := (RecordCountAfterOpen >= 0) and (RecordCountAfterOpen = 2 then
    WriteLn(Format('  AfterFetch (FetchCount = %d)', [FFetchCount]));
end;

begin
  try
    if TestQueryRecCountWithFilterAppliedBeforeOpen then
      WriteLn('PASS')
    else
      WriteLn('FAIL');
  except
    on E: Exception do
    begin
      WriteLn('FAIL - Exception Error');
      WriteLn('  E.ClassName = ', E.ClassName);
      WriteLn('    E.Message = ', E.Message);
    end;
  end;
  ReadLn;
end.
Steps:
- execute the script
- create a new console application in Delphi and paste the code
- adjust the constants cServer, cUserName and cPassword
- compile and run the example

expected: output is PASS
actual:
FAIL - RecordCount = 930
FAIL

(Tested with 5.80.0.42 with Delphi 5 and 8.1.3 with XE and XE2)

Some remarks:
- when applying the filter after open then record count is correct, but more initial fetches are done
(for testing set VerboseLevel constant to 2 and OpenBeforeApplyFilter to True)
- without QueryRecCount the record count is lower than the expected count, but this is okay, because not everything has been fetched yet. When everything has been fetched the record count is correct.
(for testing set VerboseLevel constant to 2 and QueryRecCount to False)
- the wrong record count seem somehow to depend on the amount of records, because it is 429 when you add "WHERE ID < 500" to the SQL statement (however the value is still nonsense)

Posted: Thu 29 Dec 2011 14:22
by bork
Hello

It is correct behavior. The Filter property allows to filter records on the client side. We cannot predict how many records in database will be correspond to a Filter before these records are fetched. There are several ways to resolve this issue:
1. Set the FetchAll property to True.
2. Call qr.Last before getting records count.
3. Use AddWhere('ID > 70 AND ID 70 AND ID < 80' for filtering on the server side.