Data Mapping Issue.. sometimes..

Discussion of open issues, suggestions and bugs regarding UniDAC (Universal Data Access Components) for Delphi, C++Builder, Lazarus (and FPC)
Post Reply
FredS
Posts: 211
Joined: Mon 10 Nov 2014 17:52

Data Mapping Issue.. sometimes..

Post by FredS » Fri 13 Nov 2020 20:59

I've had this come up twice, once while debugging and another time as a bug report submission from a Dual Xeon (NUMA) system.

I've never been able to reproduce it until yesterday when I tried FastMM5 and it showed up.
Once I also turned on MadExcept Leak reporting it was a breeze to reproduce.

I traced it to TCRRecordSet.GetFieldData in particular Line: 4376/4377.
On rare occasions accessing TCRFieldDesc(Field).MapRule gives an Access Violation.
Commenting those two lines out made everything work fine since I only had two Data Mappings and neither used them.

An attempt to use Breakpoints to log data using Codesite eliminated the error.
After evaluating the Mapping code I decided that using MACROS and CAST in the two SQL statements was easier.

But MapRule is never nil yet at some point it must have been freed or is pointing nowhere.. not going to attempt to try and duplicate this but you should know..

MaximG
Devart Team
Posts: 1530
Joined: Mon 06 Jul 2015 11:34

Re: Data Mapping Issue.. sometimes..

Post by MaximG » Mon 16 Nov 2020 17:08

Thank you for bringing this to our attention. Could you send us a sample project demonstrating the issue?

FredS
Posts: 211
Joined: Mon 10 Nov 2014 17:52

Re: Data Mapping Issue.. sometimes..

Post by FredS » Tue 17 Nov 2020 16:46

As stated I've simply removed 'Data Type Mapping' because I was not able to reproduce until I added FastMM5 and Leak Checking..

In short I have no incentive to go and see what could possibly be the issue. But threading would be in my list of things to consider. Here is why:
  • Reproducing this requires me to run several threaded Data Modules with db access first, yet none use 'Data Type Mapping'
  • By the time that Mapping code in GetFieldData is called all those threaded connections are terminated and back in Pooling
  • Yet simply adding a Breakpoint with CodeSite logging stops the exception, CodeSite uses a Critical section
  • That said all calling code is guaranteed to be execute in the Main thread because an assertion already checks that.
I've added a Tag to source control and can revert to reproduce at any time but am NOT willing to invest the time required to track this down..

FredS
Posts: 211
Joined: Mon 10 Nov 2014 17:52

Re: Data Mapping Issue.. sometimes..

Post by FredS » Fri 20 Nov 2020 22:16

Each time the Dataset is opened it creates a new MapRule entry for the Collection, but that is not updated.
All runs until the first memory is reallocated, mostly the string (Format) because its a managed type.

Data Type Map and Field Definition:

Code: Select all

    DataTypeMap = <
      item
        FieldName = 'FETCHHASSNAPSHOT'
        FieldType = ftBoolean
      end>
    object LookupDomainsFETCHHASSNAPSHOT: TBooleanField
      FieldName = 'FETCHHASSNAPSHOT'
      ReadOnly = True
    end      
Callstack:

Code: Select all

main thread ($12ac):
0048b21a +0eda MyApp.exe System.SysUtils            WideFormatBuf
00489d20 +0020 MyApp.exe System.SysUtils            FormatBuf
00489f79 +0099 MyApp.exe System.SysUtils            FmtStr
00489ea5 +0015 MyApp.exe System.SysUtils            Format
00489e77 +0017 MyApp.exe System.SysUtils            Format
028fba84 +0264 MyApp.exe CRAccess         4378  +16 TCRRecordSet.GetFieldData
0279e93e +017e MyApp.exe MemData          3222  +16 TData.GetField
Code Modifications and Breakpoints **:

Code: Select all

procedure TCRRecordSet.GetFieldData(Field: TFieldDesc; FieldBuf: IntPtr; Dest: IntPtr; NeedConvert: boolean);
{$IFNDEF LITE}
var
  Buf: IntPtr;
  Status: TConvertStatus;
  Value: Variant;
  Str: string;
  s : string; fd : TCRFieldDesc absolute Field;
{$ENDIF}
begin
{$IFNDEF LITE}
  if NeedConvert and (TCRFieldDesc(Field).OnDemandConverter <> nil) then begin
    Buf := GetBufForDataMappingConverter(FieldBuf, Field.DataType, Field.HasParent, True);
    s := Format('Type: %d, Addr: %x', [Ord(Field.DataType), UIntPtr(fd.MapRule)]); 
    s := Format('>> MapRule.IgnoreErrors: %s', [BoolToStr(fd.MapRule.IgnoreErrors, True)]); **
    try
      FConvertInfo.StringHeap := FStringHeap; **
      FConvertInfo.Source := Buf;
      FConvertInfo.SourceOffset := 0;
      FConvertInfo.SourceLen := Field.Length;
      FConvertInfo.SourceScale := Field.Scale;
      FConvertInfo.Dest := Dest;
      FConvertInfo.DestOffset := 0;
      FConvertInfo.DestLen := TCRFieldDesc(Field).MapLength;
      FConvertInfo.DestScale := TCRFieldDesc(Field).MapScale;
      s := Format('>> Try MapRule.Format: %s', [fd.MapRule.Format]);
      FConvertInfo.IgnoreConvertErrors := TCRFieldDesc(Field).MapRule.IgnoreErrors; **
      FConvertInfo.Format := TCRFieldDesc(Field).MapRule.Format;
      ...

Code: Select all

function TCRMapRules.AddRule(const FieldName: string;
  DBType: Word; DBLengthMin, DBLengthMax, DBScaleMin, DBScaleMax: Integer;
  DataType: Word; FieldLength, FieldScale: Integer;
  IgnoreErrors: boolean; const Format: string): TCRMapRule;
var
  Error: Exception; s : string;
begin
  Error := DBTypeInfos.Check(DBType, DBLengthMin, DBLengthMax, DBScaleMin, DBScaleMax, FieldName = '');
  Result := nil;

  if Error = nil then begin
    Result := GetMapRuleClass.Create(Self);

    s := Sysutils.Format('AddMapRule.Index: %d - Addr: %x', [Result.Index, UIntPtr(Result)]);
    Result.FieldName := FieldName; **
    ...
CodeSite:

Code: Select all

AddMapRule.Index: 0 - Addr: 1D421DC8
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format: 
AddMapRule.Index: 0 - Addr: 19795718
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format: 
AddMapRule.Index: 0 - Addr: 19795718
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format: Kaput**
Breakpoints: https://imgur.com/a/EhVx7g5

Post Reply