Reading an XLS file using CFQUERY

I haven't really seen a lot of documentation on how to read an XLS file into Coldfusion. Luckily, one of my colleagues was able to point me in the right direction. I am now here to write up a blog post for future reference.

Once you set up the new datasource in the CF Administrator (see comments below), it's really simple (at least more simple than parsing through the other file formats that I've had to do far). Basically, we just read it in as a query:

[More]

SELECT COUNT(*), The Basics

In SQL, COUNT() is an aggregate function that we can use to count the number of rows in a query. We can also use this function to figure out if there are duplicate records in a table.

[More]

More tips for SQL efficiency: boolean values in SQL

I'm always looking for ways to speed up my SQL Server queries.  Here's a tip that I used just today to speed up a search query, and I thought maybe it would be useful for all of you.

First up, make sure to include an execution plan by right-clicking on your query window and selecting "Include Actual Execution Plan"

Let's take a look at the following query:

view plain print about
1SELECT DISTINCT a.author_id, a.author_name, b.book_id, b.book_title
2FROM authors a
3INNER JOIN books b ON a.author_id = b.author_id
4WHERE a.author_id = 27
5AND b.book_published = 1
6AND b.best_seller = 1

As you can see, we have two tables, books and authors. Each of these tables are joined to show the author names for each book, and we filter by books that are published and are best sellers. This is where we'll be concentrating.

[More]

Getting the names of columns in a table

Good afternoon everyone, today were going to talk about meta-data.  Specifically data relating to tables in SQL Server.  Have you ever wanted to grab all of the column names of a table for some operation?  Well, here is a break down of a technique that can be used to do it.

The first table that we'll be working with is the "sysobjects" table.  This table contains all objects within a database.  Objects are things such as views, tables, and stored procedures.  If you want to see a full collection of this you can query this table and see it first hand:

view plain print about
1SELECT name
2FROM mydb.dbo.sysobjects
3ORDER BY name

To filter only tables you can add an xtype of "U"

view plain print about
1SELECT *
2FROM mydb.dbo.sysobjects
3WHERE xtype = 'U'
4ORDER BY name

 

Now on to column names, we can get the column names for a specific table by joining this on the "syscolumns" table. 

view plain print about
1SELECT mydb.dbo.syscolumns.name AS column_name
2FROM mydb.dbo.syscolumns
3    INNER JOIN mydb.dbo.sysobjects ON mydb.dbo.syscolumns.id = mydb.dbo.sysobjects.id
4WHERE (mydb.dbo.sysobjects.name = N'table_name')

 

There you have it, working with these metadata tables is incredible useful, good luck on using it!

SQL tips for efficiency

 Good morning folks,

Here's a couple new tips today to make life easier in SQL.  When analyzing queries there are two tools which are great for examining resource usage.  The first is the STATISTICS IO.  You can turn this on by setting the results to text, and prepending your query with the following:

view plain print about
1SET STATISTICS IO ON

This will allow you to view the table read and read-ahead data related to the query.  Once it's on it will stay on within the query window you use it in.

Here is a simple example of how it''s used:

view plain print about
1SET STATISTICS IO ON
2GO
3SELECT TOP 50
4client_id FROM sales

Whenever I use this functioanlity I use it in tandem with an execution plan.  To activate your execution plan, right click on the query window in the query analyzer, and select "Include Actual Execution Plan".  You then run your query and a new tab will appear next to the results pane called "Execution Plan".  This will show the "path" that SQL takes in deciding how a query is processed.  When you hover over icons in this window you'll find that you can see the data pertaining to each specific element of the query as well.  Keep an eye out for "CONVERT" functions and "CAST" functions in this window, as preventing these by typing your data correct can save a lot of headaches.  Additionally you can see which indexes SQL uses to grab it's data.

 

SQL CASE statements

One thing I discovered recently was the power of the SQL CASE statement. I had to do multiple joins on multiple tables and then proceed to select data from a certain field that can actually be a different field depending on certain circumstances, as one unified field. Sounds fun, right?

One way you can figure out which field to use is to return them all and take care of the display logic on the Coldfusion side of things. The problem I had with doing this was the query I was building was probably going to be used in multiple places and I didn't want to tote that Coldfusion logic everywhere when I could just handle it in the query itself.

The first thing I learned was that apparently you cannot use SQL CASE statements to decide which tables to join on. What I have concluded you do is join all the tables you need to, then use the CASE statements to figure out which fields to select or filter on.

This leads me into why I am now a big fan of CASE statements in SQL - you now have unified data to present across the pages.

[More]

Generating joined identity columns as lists in SQL

Hello everyone!  Let's talk SQL.  My names Jonathan and today I have a useful tip for a new view on queries.  Generally when someone queries two database tables that share an identity column, they join the data and end up with a collection of rows reflecting each result, essentially creating a "combined key".  To show this example more simply, take a look at the example below:

view plain print about
1SELECT c.client_id, s.sale_id
2FROM clients c
3    INNER JOIN sales s ON s.client_id = c.client_id

client_id

sale_id
1 2
3 3
3 4
4 5
3 6
1 7

Fig-1


This query outputs a "client sales" (See Fig-1) result set that has 7 rows.  In this example such a small result set is hardly a problems, but let us imagaine that we have hundreds of clients, each with thousands of sales under their belt.  This result set would suddenly be very daunting.  Post-processing the result in your choice web service would be a performance headache, as to output it someone would have to either do a query of a query to show which clients have which sales, or this would have to be broken into a single cleint table query, and then individual queries  in a loop on the web server end.

Wouldn't it be great if there was a way to greatly reduce the row count, the number of queries, and the amount of processing on the web server.  The solution below does exactly that. 

view plain print about
1SELECT c.client_id, left(sl.saleslist,len(sl.saleslist)-1) AS sale_list
2FROM clients c
3cross apply (
4                SELECT CAST(s.sale_id AS varchar) + ','
5                FROM sales s
6                WHERE s.client_id = c.client_id
7                for xml path('')
8            )sl(saleslist)

client_id

sale_id
1 2,7
3 NULL
3 3,4,6
4 5

Fig-2


The query above casts an nodeless XML result set into a varchar field and delimits it with comma's, creating a list (See Fig-2).  The last character is removed in the main select clause otherwise the last character would be a comma.

That's it!  You'll now be able to utilize new new field (in this case sale_list) in any list functions you may have. 

Create an Excel document from any ColdFusion query object

Here's an issue that has come up many times in my years of working with ColdFusion: a client wants a one-off report in Excel. I created a function that does just that, taking any CFQUERY as input and outputting the results as an Excel file that can be saved or opened locally by the user. Here's the function:

view plain print about
1<cffunction name="queryToExcel" returnType="any" output="true" hint="Given a query and an optional output file name, output the query as an Excel document using the query columns as columns in Excel.">
2&nbsp;&nbsp;&nbsp;&nbsp;<cfargument name="queryToConvert" type="query" required="true" hint="Any valid CF Query Object" />
3&nbsp;&nbsp;&nbsp;&nbsp;<cfargument name="sFileBase" type="string" required="false" default="yourfile" hint="Desired base file name. Defaults to 'yourfile'. Output filename will be the value of this variable with .xls appended." />
4&nbsp;&nbsp;&nbsp;&nbsp;<cfheader name="Content-Disposition" value="inline; filename=#sFileBase#.xls">
5&nbsp;&nbsp;&nbsp;&nbsp;<cfcontent type="application/vnd.ms-excel">
6&nbsp;&nbsp;&nbsp;&nbsp;<table border=1>
7&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<tr>
8&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<cfloop list="#arrayToList(queryToConvert.getColumnList())#" index="sColName">
9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<th><cfoutput>#sColName#</cfoutput></th>
10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</cfloop>
11&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tr>
12&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<cfloop query="queryToConvert">
13&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<tr>
14&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<cfloop list="#arrayToList(queryToConvert.getColumnList())#" index="sColName">
15&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<td><cfoutput>#queryToConvert[sColName][queryToConvert.currentRow]#</cfoutput></td>
16&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</cfloop>
17&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tr>
18&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</cfloop>
19&nbsp;&nbsp;&nbsp;&nbsp;</table>
20&nbsp;&nbsp;&nbsp;&nbsp;<cfreturn true>
21</cffunction>

Ideally you'll create your query with column names that make sense to the user. Here's an example of an invocation of the above function from a CFM page with an employee query. Our database is SQL Server, so your mileage may vary when creating user-friendly column names.

view plain print about
1<cfquery name="qEmpDetails" datasource="myDSN" >
2&nbsp;&nbsp;&nbsp;select
3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;emp_id AS [Employee ID],
4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;emp_fname AS [First Name],
5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;emp_lname AS [Last Name],
6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;emp_email AS [Employee Email]
7&nbsp;&nbsp;&nbsp;from employees
8&nbsp;&nbsp;&nbsp;order by emp_lname,emp_fname,emp_id
9</cfquery>
10
11<cfscript>
12&nbsp;&nbsp;&nbsp;&nbsp;// Note my components are stored in the folder /shared/components;
13
&nbsp;&nbsp;&nbsp;&nbsp;// The component name is 'Utils.cfc'
14
&nbsp;&nbsp;&nbsp;&nbsp;utilsObj = createObject('component', 'shared.components.Utils');
15&nbsp;&nbsp;&nbsp;&nbsp;utilsObj.queryToExcel(qSupDetails,'supplier_details');
16
</cfscript>

I hope this helps somebody who needs to do occasional Excel exports. Note that Excel has a limit of 65,536 rows per sheet; a nice exercise would be to adapt this tool to check the number of rows in the query and be more flexible in its output options.

Previous Entries