Categories
Cloud FireDAC REST Source Code

API Limits with #FDEC

API Limits with FireDAC Enterprise ConnectorsThe FireDAC Enterprise Connectors (#FDEC) by CData and Embarcadero make it really easy to work with various APIs just like you would any SQL database. For example if you want to publish the results of a query to a Google Sheet (which I find incredibly useful) then it is just a few FireDAC components and you are off to the races. You might run into an API limit though.

What is an API limit? Most rest services have a limit to how often a client can call a specific API within a certain amount of time. Google calls this their usage limit:

This version of the Google Sheets API has a limit of 500 requests per 100 seconds per project, and 100 requests per 100 seconds per user. Limits for reads and writes are tracked separately. There is no daily usage limit.

That may seem like a lot, but I found I was running into that limit pretty quick once I moved my project into production. Luckily FireDAC and the FireDAC Enterprise Connectors have a simple workaround: Batch Processing.

Using the Array DML features of FireDAC you can batch multiple DML (Data Manipulation Language) operations into a single API call. The FDEC Google Sheets documentation from CData doesn’t cover Array DML, but the component supports this (they are updating the documentation). The Elasticsearch documentation does cover Batch Processing with an example, and I’ve used this with Sheets and it works great!

Bulk Insert

The following example prepares a single batch that inserts records in bulk.

FDConnection1.ResourceOptions.ServerOutput := True;
FDQuery1.SQL.Text := 'insert into Account values (:Name, :Id )';
FDQuery1.Params.ArraySize := 100;
FDQuery1.Params[0].AsStrings[0]:= 'MyName1';
FDQuery1.Params[1].AsStrings[0]:= 'MyId1';

//next statement
FDQuery1.Params[0].AsStrings[1]:= 'MyName2';
FDQuery1.Params[1].AsStrings[1]:= 'MyId2';
...

FDQuery1.Execute(FDQuery1.Params.ArraySize);
ShowMessage(IntToStr(FDQuery1.RowsAffected));

To retrieve the Ids of the new records, query the LastResultInfo#TEMP table:

sName := FDQuery1.Open('SELECT * FROM [LastResultInfo#TEMP]');

Bulk Update

The following example prepares a single batch that inserts records in bulk.

FDQuery1.SQL.Text := 'update Account set Name = :Name WHERE Id = :Id';
FDQuery1.Params.ArraySize := 100;
FDQuery1.Params[0].AsStrings[0]:= 'Floppy Disks';
FDQuery1.Params[1].AsStrings[0]:= 'Id1';

//next statement
FDQuery1.Params[0].AsStrings[1]:= 'Jon Doe';
FDQuery1.Params[1].AsStrings[1]:= 'Id2';
...

FDQuery1.Execute(FDQuery.Params.ArraySize);
ShowMessage(IntToStr(FDQuery1.RowsAffected));

Bulk Delete

The following example prepares a single batch that inserts records in bulk:

FDQuery1.SQL.Text := 'delete Account where Id = :Id';
FDQuery1.Params.ArraySize := 100;
FDQuery1.Params[0].AsStrings[0]:= 'MyId1';

//next statement
FDQuery1.Params[0].AsStrings[1]:= 'MyId2';
...

FDQuery1.Execute(FDQuery.Params.ArraySize);
ShowMessage(IntToStr(FDQuery1.RowsAffected));

If you want to learn more about Array DML check out these videos:

Array DML Skill Sprint with Pawel Glowacki

FireDAC in Depth with Cary Jensen

Also check out Cary Jensen’s book on the topic of FireDAC in Depth.

Categories
News

Celebrating Five Amazing Years with Embarcadero

This month I am celebrating my five-year anniversary with Embarcadero Technologies. It’s been an amazing adventure and I am looking forward to more. I saw a graphic similar to the following today and it reminded me how great it is to be doing what I am doing (plus I love Venn diagrams).

Loving what I do at Embarcadero

I truly do love what I am doing, and I love the people I work with. Everyone I work with inside Embarcadero is amazing, plus I get to work closely with the MVPs, Tech Partners, and all the amazing members of the community. Most of what I do is stuff I used to do as an MVP before coming to Embarcadero. It was essentially my hobby. I would even take vacation time and spend my own money to travel to speak at conferences.

I frequently talk to developers who express their appreciation of the work I do as a developer advocate – I love training and sharing what I know. Not to mention all the gratitude I hear from developers for the productivity they get from Embarcadero’s tools. The reality is Embarcadero’s tools help developers make the world a better place.

RAD Studio speaks AndroidI started right before the release of RAD Studio XE5 in 2013, which added support for Android. iOS and macOS were great, but once RAD Studio had Android it was a gamechanger. So many people just added Android as a target to their iOS apps, made a few layout tweaks, and they had a native Android app!

Since then we’ve added so many great features, plus Linux and the new Free Community Edition!

When it comes to monetary pay I’m really looking for a way to take care of my family. Beyond that, my job pays a lot in satisfaction because I believe in what I do. It certainly helps that Software Developer was rated the #1 job this year. I’m not directly a developer, but software development is a big part of my job, and I work with a lot of amazing developers.

 

It’s been an amazing five years. Despite a few bumps I am so happy for where we are going, what I do here, and where Delphi, RAD Studio, and Embarcadero are headed. It is always rough when you go through a merger, but the new Community Edition is a direct result of the changes in philosophy that Idera brought. And now I get to work with Sencha, RanorexGurock, and others that make up the Idera family.

I’m looking forward to many more years doing what I love here with Embarcadero!

ILoveDelphi

Categories
Source Code Tools

Use the Source!

One of the great things about Delphi is not only is it written in Delphi (mostly), but it ships with the VCL, RTL, & FMX source code. You can use this source code in lots of different ways, which I’ll cover in a bit, but sometimes it is a matter of finding the source file with the code you want. There are over 2,234 Delphi source files in the source folder, so it can take a while to find the right file if you don’t know where to look. Not to mention the 1,711 C files, and a few thousand other assorted files.

Source Folder Files

I used to use various GREP and full text searching tools to find the code I was looking for, but then I realized Windows 10 (and earlier versions) has a search function built into it, but you need to make a few configuration changes to use it effectively. So I thought I would outline those for you.

Your source folder is usually located in
C:\Program Files (x86)\Embarcadero\Studio\19.0
but it might be different depending on your installation. I usually Pin it to Quick Access, which makes it easy to access since I find I’m there a lot.

Source Folder

Windows Search and Indexing Options

The next thing to do is configure your Indexing Options. This is what makes it quick to search for files you need. You can find Indexing Options in Control Panel, or just runcontrol.exe srchadmin.dll.

control.exe srchadmin.dll

There are two steps to get the most out of Windows search for your source code. The first is you need to activate Windows search for the file extensions you want, and the second it to tell it to index your folders.

Indexing Options

To add PAS files to indexing click Advanced and go to File Types. You can just type PAS in the list of file types and it will find it for you. You don’t need to add a new extension. PAS files are already listed, but just not indexed by default. Just put a check next to it, and change the “How should this file be indexed?” to “Index Properties and File Contents” that will index all the text contents of the file

Indexing File Types

By default all your user folders are indexed, but Source is in your Program Files folder, so you need to explicitly add it. From the main Indexing Options window click Modify and from there you can add any folder you want to search.

Indexed Locations - Source

It takes a while a while to build and update the index after this change. You just gave it 2,000 more files to index. It has to read all the text out of them, which takes a while. Once it is done then just go to your source folder and use the search box to quickly find the code you are looking for.

Search Results

Everything Search

Everything Software IconAnother great tool to use, to search by file name is the Everything desktop search engine by Voidtools. The thing I love about using Everything is it searches all the file names on your system quickly. The difference between it and Windows Search is Everything only searches the file names, but it searches all the file names.

Everything Search

I find I usually remember the name of a file, but not exactly where I left it. This makes Everything indispensable for me.

Using the Source

So how useful is it that Delphi, RAD Studio and C++Builder includes all this source code? What all can you do with it?

  • Finding that function – You know how I said I can remember the name of a file, but not where I put it? I do the same things with classes and functions. I can remember the name of the method or class, but not which unit it is in. Being able to search all of the source instantaneously to find the source file you need is a huge help.
  • Learning – Just reading source isn’t the best way to learn to program, but having access to the source code is a great way to gain a deeper understanding.
  • Debugging – Sometimes your program doesn’t behave the way you expect it to. When all you have is a blackbox then you are unable to figure out what is happening behind the scenes. When you have the source code you can not only see how that method is implemented, but also debug into the source code, using all the great debugger features.
  • Adding New Features – Maybe there is a new API that was just released, or you need to access an obscure API or 3rd party feature. Since you have access to all the source you can see how other similar APIs are accessed and use that as a roadmap.
  • Fixing or Changing Functionality – One man’s bug is another man’s feature. Sometimes there are bugs you need to fix in the shipping source code, other times you just want to change the way things work. For small changes I just copy the source file out into my project directory. Then my code will use my modifications instead of the original implementation. While you can’t redistribute changed source code, you can compile the code into your program. If you do need to share your changes or fixes you can share a change-set that only contains your changes.

What else do you use the source for?

Categories
Funny Source Code

3D Credits Scroll with Delphi

A little fun with Delphi for today’s Star Wars day!

3D Credits Scroll with Delphi - May The Fourth

This is a pretty simple 3D form with a 3D layout at an angle, and then a 2D layout with text and images is animated up the 3D layout. The only code is populating the labels as the animation kicks off automatically.

3D Scroll - MayTheFourth-StructureDownload the code and run the full credit scroll, or change the message to share with your friends. It is FireMonkey, and while I only tested it on Windows, it should work on Android, iOS, macOS, and even Linux if you are running FMX Linux.

CreditScroll3D Source for 10.2.3 Delphi

May the Fourth be with You!

Categories
News podcast Video podCast

LIVE! With Boian Mitov of Mitov Software

Join us Tuesday, March 20th, for a LIVE conversation with Boian Mitov of Mitov software as we talk about Arduino, Visuino, IoT, Industrial Automation, AI and more!

(We had to reschedule due to technical difficulties)

11 AM Mountain Daylight Time

 

Categories
News podcast Video podCast

Join us for a LIVE Video Episode of the Podcast at Delphi.org

Tomorrow will be a live episode of the Podcast at Delphi.org

You can join us live on YouTube with your questions and comments, or catch the replay posted here. Stay tuned tomorrow for details on how to join.

This is our first time with this new format and would love to get your feedback. We’ll also appreciate your patience as we work out the technical details.

Categories
Article News

Get Ready for the New Programmers!

U.S. News an World Report just ranked Software Developer as the best job for 2018. They use data from the U.S. Bureau of Labor and Statistics to rank jobs based on pay, job security, mental engagement, stress, room to advance, satisfaction, and work-life balance. Since you are here, you most likely agree with that ranking and are surprised it took so long for someone else to recognize it.

Software Developer #1 Best Job

This means we will see a lot of people interesting in software development. Anyone looking for a better job is likely to start at the top of that list and work their way down until they find one they are interested in. Not to mention everyone who sees the headlines about Software Developer replacing Dentist as the #1 Best Job. Whatever the reason, software developers will get a lot more attention.

Everyone is Focused on Software Developers

When I’m talking to people about career advice I think it is more important to choose a career that suits the individual (internal factors) than basing the decision purely on external factors like pay, etc. That being said, I honestly believe Software Development is only going to get more important. Going forward, software development and related jobs (many of which aren’t even invented yet) will consume the many of the other jobs as automation and artificial intelligence take over more aspects of our life. It all depends on which side of the automation revolution you want to be.

 

When I was really young (like 3rd grade) I knew I wanted to program computers for a living. A family friend told me that I should look for a different job because by the time I entered the career market computers would be programming themselves and there would be no jobs. I remember thinking once that happens there would be no jobs, and someone would need to teach the computers how to program themselves better.

Just recently I was in Tokyo for the 10.2 launch event. I was talking to members of the press, and one of them made a similar assertion “What is the point of releasing better developer tools when soon computers will be programming themselves?” I shared the story from when I was a kid and said that “Yes, AI is automating and consuming other jobs, but the programmer will be the last to go. Once AI’s no longer need humans to make them better there will be no jobs for anyone!”

Congratulations, you choose wisely.

So what does this mean for you, as a software developer today? Congratulation’s you choose wisely! But be prepared for a lot of people to come to you for career advice. Remember that Delphi offers a great Free Starter edition, and Embarcadero Academy is full of content for people learning to program.

Categories
Article News

Law of the Instrument and Curse of the Programmer

If you aren’t familiar with the Law of the Instrument, otherwise known as Maslow’s hammer/gavel, or the golden hammer it is often expressed as

If your only tool is a hammer, you treat everything as like a nail.

My understanding is that the law of the instrument means you are limited by instruments or tools you know how to use. For example, if you have a screw, some wood, and a hammer, then you might successfully get the screw into the wood, but a screwdriver would be a better alternative.

Law of the instrument
Image from Pixabay by Steve Buissinne

The law of the instrument also means an obsession with the perfection of the instruments you know. I remember back in the day when I was convinced there was no reason to bother with any other programming languages because Delphi was the best. Now I’ve spent some time using a lot of other programming languages, and so I can confidently say Delphi is the best, while I can see the value and use of other programming languages.

I believe it is worthwhile learning about new technologies, frameworks, languages, or methodologies. Then you can pick the correct one for the job. This doesn’t mean you need to be an expert in all of them, but you should know enough that you are confident in your choice.

The reverse of this is the obsession to chase new and exciting technologies and recreate things every few years. This keeps the developers entertained, but doesn’t really provide business value. Again I believe Delphi does a good job with this as it respects your existing code while moving forward to new platforms, features, and frameworks.

So what is the Curse of the Programmer?

When I’m talking to other programmers I see two behaviors. The first is, every problem they encounter in life (at work and beyond) they respond with “I could write a program to do this,” or some variation. By extension, they also cast a critical eye toward any software system (even those developed by themselves) to see how to do them better. This results in a huge backlog of projects that they create to fix problems, fix a problem better, or just out of curiosity to see if they can.

This is similar to the Law of the Instrument, but I see it more as your learning the flexibility and power of programming results in your seeing many opportunities to apply it. I’ve talked to people in other industries, and I think the general tendency is fairly universal, it is just that programming is (in my opinion) so much more powerful and flexible than many other applied technologies.

The second behavior, which is something to be more cautious about falling into, is the urge to create a “library” or “framework” instead of finishing the program at hand. For example, you are creating a program to solve a problem, and in the process, you create series of libraries just in case you need to solve similar problems.

There is value in having reusable libraries, functions, components, and frameworks. The trick is to not let the creation of them get in the way of shipping. The best way I’ve found to deal with this is to only create the library when you need it. Write your code with the appropriate level of coupling to solve the problem at hand. When you need to reuse a bit of it elsewhere, consider refactoring it into something reusable. Then as you use it in more places you can keep refactoring it and expanding it until you have a full blown framework.

How do you see the Curse of the Programmer in your life? What do you use to prevent every project from spawning a series of reusable frameworks?

Categories
News

Some of my New Year’s #CodingResolutions

The joke goes that, when asked, the software developer responds that their New Year’s Resolution is “7680×4320” or 8K. Joking aside, tradition has it that at the turn of the year we evaluate our lives and look for areas of improvement. Here are a few of my software development related New Year’s #CodingResolutions!

  • Unit Test More: DUnitX and TestInsight being my tools of choice. Most of the code I write is integration code, integrating component A with control B, and I use that as a justification to not unit test it. But when I do write library or component code, I try to create a unit test for it. I find it very useful to do so and am always working to be more consistent.
  • Take a course on Embarcadero Academy: There are a few free courses, as well as some premium courses from the top trainers in the community. Something there for everyone to learn from.
  • Get better at using version control on personal projects: Version control is a must when you are collaborating with others on a project. I remember the first development job I had where I was working with other developers. We emailed code back and forward. We used the archived history of emails if we needed to roll back to an earlier version. The lead developer maintained the “authority.” Needless to say, this was a mess and didn’t work very well. RAD Studio has a great history feature that automatically backs up the last few versions of your files when you save, and you can set it to save on execution (a great idea), so it might seem like version control isn’t as necessary on non-collaborative projects. This just isn’t true. The IDE will integrate into today’s top version control systems, so there is no excuse not to. I signed up for a paid Developer account on GitHub, which is where I keep all my projects. BitBucket is another popular choice, which offers private repositories for free, but charges you for collaborators (the opposite model of GitHub). BitBucket offers Mercurial too. Which system do you use?
  • Setup my own Continuous Integration server: If you don’t have an automated, repeatable build process, then you don’t have a release. Designating one of your developer’s computers (even if that is you) as the build server is not a valid solution. If you are going to have a Build Server, then you might as well have CI (with automated Unit Tests!) so you know your build is always good. Craig Chapman provided a recent webinar on setting up your own Continuous Integration system with SVN & Jenkins. Jenkins is a popular solution for CI that even offers a RAD Studio plugin, but if you want to simplify things, take a look at Continua-CI from VSoft.
  • Automate more with DevOps: Version Control and Continuous Integration are the first steps in DevOps as a developer, but there is more. Everything that can be automated should be automated.
  • Pay attention to Warnings, Hints, Static Code Analysis, Audits and Metrics: One of the beauties of using a compiled language is we have a compiler that will immediately detect a huge number of errors, which we must fix if we want to even use or test our programs. The compiler also provides a number of Hints and Warnings that frequently indicate your program might not function exactly like you intended. I have a policy of always addressing Hints and Warnings too. I’m also a big fan of FixInsight, which really expands these compiler messages in so many useful ways. Our favorite IDE also includes more options to keep your code fresh and maintainable: Code Toxicity and Audits and Metrics. The Code Toxicity is available in all editions and provides some simple static code analysis for things like Cyclomatic Complexity. Keeping an eye on these is a good way to make sure your code is maintainable and less likely to have unintended behaviors. The Audits and Metrics takes this to the next level with a lot more detail. And it’s included in Enterprise and Architect editions.RAD Studio Audits
  • More focus on best practices and software security.
  • Do more with RAD Server. I’ve done a lot of building REST clients and they are great. I’ve done some basics of building REST servers but would like to do more. RAD Server is a great solution for building REST services.
  • Deploy RAD Server to the Cloud. I’ve done a little with Amazon Cloud Services and that is what I am thinking about doing, but there are a lot of other great cloud hosting options out there to consider.
  • Keep Exploring the Amazing 3rd Party components from all the great Technology Partners. The GetIt Package Manager makes it easy to install a new component or library. 
  • Build some Neural Network and Artificial Intelligence solutions with Delphi. This is something I’ve always been interested in but never implemented. There are so many advances in this field right now, which makes it even more exciting. Additionally, there are so many promising libraries available for Delphi. Boian Mitov of Mitov Software has IntelligenceLab. Dew Research has a great looking math engine. And then RiverSoft has a serial of libraries for Genetic AlgorithmsFuzzy Logic, and an Inference Engine. You can check out a video about IntelligenceLab today. I’d love to see an IDE add-in that uses neural networks for static code analysis and help with writing code.
  • Beyond all of that, I want Blog, Podcast, and create more training videos.

How about you? What are your New Year’s #CodingResolutions?

Categories
Conferences News

Robert “Uncle Bob” Martin is Speaking at CodeRage XII

Here in the Delphi community, we all know Dr. Bob Swart, and CodeRage wouldn’t be complete without him (he has a session this year on Customizing DataSnap Method Output, don’t worry.). This year we are also hearing from Uncle Bob, the author of Clean Code – A Handbook of Agile Software Craftsmanship. This is one of those “must reads” that all software developers should have on their bookshelf.

I had an opportunity to see Uncle Bob speak a few years ago, and it was very memorable. This session will be like no others you’ve seen. He is speaking on “The Clean Coder – An Introduction to Software Professionalism.” His session and the live Q&A will be available only during the live CodeRage session, so be sure to join Uncle Bob for this amazing session!

Register now!

Robert "Uncle Bob" Martin