Page 1 of 1

Want function ResultValue string to return char rather than MEMO

Posted: Tue 14 Jul 2015 15:57
by NoComprende
Hi, My TLiteUserFunction looks up a char(20) value in an sqlite table and then assigns this to ResultValue but this value appears in the query result as MEMO. How do I get it to appear as CHAR?

Re: Want function ResultValue string to return char rather than MEMO

Posted: Wed 15 Jul 2015 15:18
by AlexP
Hello,

If you are using UserFunction, concatenation, aggregating functions, etc., SQLite returns the Unknown type for such fields. For correct data display, you should use DataTypeMapping http://www.devart.com/litedac/docs/data ... apping.htm .

Re: Want function ResultValue string to return char rather than MEMO

Posted: Wed 15 Jul 2015 16:46
by NoComprende
AlexP wrote:Hello,

If you are using UserFunction, concatenation, aggregating functions, etc., SQLite returns the Unknown type for such fields. For correct data display, you should use DataTypeMapping http://www.devart.com/litedac/docs/data ... apping.htm .
I had a look at that page Alex but I'm not really any the wiser. Are you sure it's an SQLite problem? It seems to me that all strings assigned to devart's ResultValue will appear as MEMO. If my user function does nothing more than assign ResultValue="X" then it's still displayed as a MEMO.

Couldn't devart code have dealt with this if the user was allowed to define the return type in TLiteUserFunction? I would also have liked to define the TLiteUserFunction as SQLITE_DETERMINISTIC where appropriate.

PS I'm also having difficulty returning Null values. When I assign ResultValue=Null in a function (call it RealFct) that returns a REAL value the result is displayed as -1. If a query displaying RealFct has a Null (or -1) value in the first row then the entire column is truncated to an integer. Again would this not be easy dealt with if the user was allowed to define the return type in TLiteUserFunction?

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 09:22
by AlexP
As I wrote earlier, for fields that are not real fields in a table (as well as for some of constant fields) SQLite returns Unknown. So we interpret these fields as ftMemo.
Please provide a sample of your user function that returns incorrect data on returning Null values.

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 10:03
by NoComprende
create table Tbl (ID int);
insert into Tbl values (1),(2);

Define a TLiteUserFunction Fct that points to function Fct with 1 int parameter & FctExecute as follows

void __fastcall FctExecute(TObject *Sender,TDAParams *Params,
Variant &ResultValue)
{
if (Params->Items[0]->AsInteger==1) ResultValue=Null;
else ResultValue=2.5;
}
//---------------------------------------------------------------------------

select ID,Fct(ID) from Tbl order by ID;
returns
1 | -1
2 | 2

and

select ID,Fct(ID) from Tbl order by ID desc;

returns

2 | 2.5
1 | -1

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 11:36
by AlexP
The described problem is not reproduced. On the latest SQLite version 2.5.17 correct values are returned if you set mapping rules before query opening in the following way:

Code: Select all

  LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID)', ftFloat);

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 12:38
by NoComprende
Fair enough Alex, but don't you think it would be less cryptic if you could select ftFloat as the return type from a drop down list in the TLiteUserFunction properties?

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 13:15
by AlexP
API sqlite3_create_function doesn't allow to set the result type returned by a function. Therefore we can't implement selection of the result type in our components.

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 14:01
by NoComprende
What I meant was the user selects the return type from a drop down list in the TLiteUserFunction and devart's code executes the cryptic code to implement the data map. I've no idea if that's possible but the above is hardly obvious. Nowhere in that Data Type Mapping page you directed me to earlier does it mention mapping functions.

Also, the syntax itself is confusing. e.g. If I execute the code as you've typed it above everything works just as you suggested but I defined that function to have a parameter i (not ID) yet it doesn't work if I execute the code

LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(i)', ftFloat);

Does this mean that I'd have to implement a mapping for every different parameter I called Fct with?

Having just tested it I find that if I execute 'select ID, Fct(ID/3+ID)' it doesn't return the same result (it should because ID/3 returns 0 because of integer division). It only returns them as ftFloat if I execute the statement

LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID/3+ID)',ftFloat);

????????????????????????????????????????????????????????????????

Re: Want function ResultValue string to return char rather than MEMO

Posted: Thu 16 Jul 2015 14:05
by NoComprende
AlexP wrote:API sqlite3_create_function doesn't allow to set the result type returned by a function. Therefore we can't implement selection of the result type in our components.
I also don't understand this. I'm assuming that it's devarts code that maps the SQLite result to the data mapping target field type after it receives the result. Is that not the case?

Re: Want function ResultValue string to return char rather than MEMO

Posted: Fri 17 Jul 2015 10:01
by AlexP
DataTypeMapping is applied to fields of RecordSet and is not related to SQLite user functions in any way. We can't affect the inner implementation of SQLite user functions. The remainder of an integer field division will always be discarded, not depending on where the division occurs, i.e. "Select ID/3" will also return 0.

Re: Want function ResultValue string to return char rather than MEMO

Posted: Fri 17 Jul 2015 13:13
by NoComprende
I think you're picking me up wrong here Alex. I'm aware of the rules for integer division. I merely chose that example to illustrate the fact that the data mapping only worked if the 'Fct(ID)' string in your

LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID)', ftFloat);

command exactly matched the string in the SQLite select statement. That is to say your data map example won't work for any other call of Fct.

So, if I implement your data map LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID)', ftFloat); and then execute the statement 'select ID,Fct(ID),Fct(ID+0)' the result will be

1 | -1 | -1
2 | 2.5 | 2

However, if I do this
LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID)', ftFloat);
LiteQuery1.DataTypeMap.AddFieldNameRule('Fct(ID+0)', ftFloat);
and then execute the statement 'select ID,Fct(ID),Fct(ID+0)' the result will be

1| -1 | -1
2| 2.5 | 2.5

So, my first point is that unless the user is willing to implement a data map for every syntactically different call of Fct then your "solution" isn't a solution.

My second point is this. I don't think SQLite is returning 2.5 truncated to 2, I think it's the way devart's code is picking the result up. My guess is, because there's no data map in place, devart's code must be interpreting the result in the first row (i.e. -1 - and I still don't know why Null is returned as -1) as an integer and then assuming it's an integer column. It then retrieves from SQLite the result of that column in subsequent rows as an integer value.

If I'm guessing correctly above then that brings me to my main point, why can't it be implemented as I suggested earlier. Why can't devart's code look up the properties of the TLiteUserFct Fct, note that the user selected the return type ftFloat and retrieve the result from SQLite as a double?

Re: Want function ResultValue string to return char rather than MEMO

Posted: Mon 20 Jul 2015 07:13
by NoComprende
Over the weekend I had a look at the FireDAC equivalent of the LiteUserFunction. Not a lot of difference between the two. The following applies to FireDAC but I'm guessing it also applies to LiteDAC.

To return a null value do nothing. Presumably the result value is already a null. Like LiteDAC, if the value returned by the function in the first row is a null it returns the entire column as a WideString or Memo (depending on your settings). In FireDAC though you can append a type to the column alias e.g.

select Fct(ID+6) as 'FctResult::FLOAT';

will ensure the column is of type float regardless of the value in the first row. That's certainly easier than having to define a data mapping for every query and every syntactically different call of Fct used by the query. Neither solution compares with the simplicity of having a TLiteUserFunction property ReturnType where you just select the type from a drop down list. A default ReturnType of ftUnknown would mean existing code would work as is.

Re: Want function ResultValue string to return char rather than MEMO

Posted: Mon 20 Jul 2015 09:43
by AlexP
The sample below demonstrates all the cases described above: both using field name and its type. Mapping of fields like in FireDAC is not implemented in LiteDAC. You can leave a suggestion to extend functionality at our UserVoice page: https://devart.uservoice.com/forums/104 ... 909-common .

Code: Select all

program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  SysUtils, Variants, DB, DBAccess, LiteAccess, LiteDataTypeMap;

type
  TMyClass = class
  private
    FLiteConnection: TLiteConnection;
    FLiteQuery: TLiteQuery;
    FLiteUserFunction: TLiteUserFunction;
  public
    constructor Create;
    destructor Destroy; override;

    procedure Open(const SQL: string);
    procedure ClearRule;
    procedure SetRule(const FiledName: String; DestType: TFieldType); overload;
    procedure SetRule(FiledType: SmallInt; DestType: TFieldType); overload;
    procedure onExecute(Sender: TObject; Params: TDAParams; var ResultValue: Variant);
  end;

{ TMyClass }

constructor TMyClass.Create;
begin
  FLiteConnection := TLiteConnection.Create(nil);
  FLiteConnection.Database := ':memory:';
  FLiteConnection.ExecSQL('create table Tbl (ID int)');
  FLiteConnection.ExecSQL('insert into Tbl values (1),(2)');

  FLiteQuery := TLiteQuery.Create(nil);
  FLiteQuery.Connection := FLiteConnection;

  FLiteUserFunction := TLiteUserFunction.Create(nil);
  FLiteUserFunction.Connection := FLiteConnection;
  FLiteUserFunction.FunctionName := 'Fct';
  FLiteUserFunction.Params.CreateParam(ftFloat, 'p1', ptInput);
  FLiteUserFunction.OnExecute := onExecute;



end;

destructor TMyClass.Destroy;
begin
  FLiteUserFunction.Free;
  FLiteQuery.Free;
  FLiteConnection.Free;

  inherited;
end;

procedure TMyClass.Open(const SQL: string);
var
  i: integer;
begin
  FLiteQuery.Close;
  FLiteQuery.SQL.Text := SQL;
  FLiteQuery.Open;

  while not FLiteQuery.Eof do begin
    for i := 0 to FLiteQuery.FieldCount - 1 do begin
      if FLiteQuery.Fields[i].IsNull then
        Write('Null' + #9)
      else
        Write(FloatToStr(FLiteQuery.Fields[i].AsFloat) + #9);
    end;
    Write(#13#10);
    FLiteQuery.Next
  end;
end;

procedure TMyClass.ClearRule;
begin
  FLiteQuery.DataTypeMap.Clear;
end;

procedure TMyClass.SetRule(const FiledName: String; DestType: TFieldType);
begin
  FLiteQuery.DataTypeMap.AddFieldNameRule(FiledName, DestType);
end;

procedure TMyClass.SetRule(FiledType: SmallInt; DestType: TFieldType);
begin
  FLiteQuery.DataTypeMap.AddDBTypeRule(FiledType, DestType);
end;

procedure TMyClass.onExecute(Sender: TObject; Params: TDAParams;
  var ResultValue: Variant);
begin
  if Params.Items[0].AsInteger = 1 then
    ResultValue := Null
  else
    ResultValue := 2.5;
end;

var
  MyClass: TMyClass;
begin
  MyClass := TMyClass.Create;
  try
    MyClass.Open('SELECT ID/3 FROM Tbl');
    //returns
    //  0
    //  0
    //Use FieldName  Rule
    writeln;
    MyClass.SetRule('FctField', ftFloat);
    MyClass.Open('select ID, Fct(ID) as FctField from Tbl order by ID');
    //returns
    //  1   Null
    //  2   2.5
    writeln;
    MyClass.Open('select ID, Fct(ID)  as FctField from Tbl order by ID desc');
    //  2   2.5
    //  1   Null

    //Use DBType Rule
    writeln;
    MyClass.ClearRule;
    MyClass.SetRule(liteNull, ftFloat);
    MyClass.Open('select ID, Fct(ID) as FctField from Tbl order by ID');
    //returns
    //  1   Null
    //  2   2.5
    writeln;
    MyClass.Open('select ID, Fct(ID)  as FctField from Tbl order by ID desc');
    //  2   2.5
    //  1   Null

    //Use DBType Rule both fields
    writeln;
    MyClass.ClearRule;
    MyClass.SetRule(liteNull, ftFloat);
    MyClass.Open('select ID,Fct(ID),Fct(ID+0) from Tbl');
    //returns
    //  1   Null    Null
    //  2   2.5    2.5
  finally
    MyClass.Free;
    readln;
  end;
end.

Re: Want function ResultValue string to return char rather than MEMO

Posted: Mon 20 Jul 2015 11:38
by NoComprende
While I admire your coding skills Alex that is one rigmarole to have to go through to prevent a query from returning the wrong field type. If Devart are happy with that who am I to deprive LiteDAC users of such joy.